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Introduction 



Naturalization Test Redesign Project: Civics Item Selection Analysis provides an 
overview of the development of content items for the U.S. history and government 
(civics) portion of the redesigned naturalization test. This document also reviews the 
process used to gather and analyze data from multiple studies to determine which civics 
test items to maintain, revise, or eliminate. 

The data presented in this document was compiled from two sources: a Pilot Study and a 
Supplemental Study. The Pilot Study serves as the primary source of data and was 
conducted at ten randomly selected U.S. Citizenship and Immigration Services (USCIS) 
offices throughout the nation. In the Pilot Study, applicants who chose to participate had 
ample time to study the civics questions for the naturalization test. 

Taking into account the regulatory mandate requiring “due consideration” (8 CFR 
312.2(c)(2)), a Supplemental Study was conducted to identify items that would be 
appropriate for low and high-beginning students. The Supplemental Study was conducted 
at English as a second language (ESL)/citizenship classes where prospective applicants 
were pursuing classroom education to prepare for the naturalization test and interview. In 
this study, USCIS focused on lower-level adult civics and citizenship classes. The 
purpose was to examine how individuals who were preparing for the English and civics 
portions of the naturalization test would compare with applicants who had already filed 
for naturalization and were, therefore, more likely ready to take the test. Data from this 
study provided USCIS with an indicator to determine item difficulty for a specific 
population. 

Lastly, this document presents performance ranges on individual test items for both 
studies as well as selection codes to indicate the rationale for keeping an item “as is,” 
revising the item, or discarding the item completely. 
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Background 



The need for a new naturalization test was formally stated in several studies in the late 
1990s that summarized issues of concern related to the administration and content of the 
current naturalization test. Studies by Coopers & Lybrand, the U.S. Commission on 
Immigration Reform, and the U.S. Department of Justice’s Office of Inspector General 
all commented on lack of standardization as an important issue to address. In addition, 
the U.S. Commission on Immigration Reform’s 1997 Report to Congress stated that the 
former Immigration and Naturalization Service (INS) should conduct a full review of the 
content of the naturalization test and develop a more meaningful test. 

The Office of Citizenship, a component within USCIS, was charged in April 2005 with 
developing a process to redesign the current naturalization test. The primary goal of the 
redesign process was to create a test that is fair, standardized, and meaningful to 
naturalization applicants by January 2007, with implementation beginning in October 
2008. 

Fair 

USCIS developed a naturalization test that is fair for all test takers. Fairness involves 
ensuring that the test is valid and also free of biases and barriers that may inappropriately 
affect test takers’ performance. Fairness regarding the redesigned test will be achieved 
through standardization of content and uniform test administration. 

Meaningful 

The second criterion of the redesign initiative was to create a testing process, and test 
content, that is meaningful for applicants. USCIS envisioned enhancing the test’s 
meaningfulness by encouraging civic learning through the development of concept-based 
test items and educational study materials provided through the immigration process. 
These materials would focus on concepts such as the rights and responsibilities of U.S. 
citizenship, rule of law, and inalienable rights. Immigrants will be able to use the 
knowledge acquired through either self-study or through classroom education to more 
effectively integrate themselves into their communities. To meet this need, the USCIS 
Office of Citizenship will produce educational materials to accompany the redesigned 
test. 

USCIS also recognized that infusing meaning required an understanding of the 
educational needs of those individuals pursuing U.S. citizenship. The test redesign 
process involved the selection of items adaptable to learners’ cognitive and linguistic 
abilities and that encouraged active learning and participation. 

Baseline Parameters 

With two clear objectives in mind — developing a fair and meaningful test — USCIS 
followed certain parameters to guide its redesign efforts. The first parameter was the level 
of difficulty. The redesigned test was developed to maintain the same level of difficulty 
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as the current test. To accomplish this, USCIS conducted a record study to determine the 
pass rate for the civics test and the pass rate for the three components of the English test: 
speaking, reading, and writing. With the acquired data, and by utilizing the U.S. 
Department of Education’s National Reporting System for Adult Education (NRS) to 
provide a framework and target test development level for non-native speakers of 
English, USCIS determined a linguistic and cognitive level approximation of the current 
test. (Information on the NRS can be accessed at http://www.nrsweb.org/) 

The second parameter that guided test development was the statutory requirements set 
forth in the Immigration and Nationality Act (INA). Section 312 of the INA provides “no 
person shall be naturalized as a citizen of the United States upon his own application who 
cannot demonstrate an understanding of the English language, including an ability to 
read, write and speak words in ordinary usage in the English language: Provided, that the 
requirements of this paragraph relating to the ability to read and write shall be met if the 
applicant can read or write simple words and phrases to the end that a reasonable test of 
his literacy shall be made and that no extraordinary or unreasonable conditions be 
imposed on the applicant.” With regard to the civics portion of the test, the law states in 
section 312(a)(2) that an applicant shall not be naturalized as a citizen if he cannot 
demonstrate “a knowledge and understanding of the fundamentals of the history, and of 
the principles and form of government of the United States.” In addition, the INA 
provides information on applicant exemptions based on physical and medical conditions 
as well as certain exemptions for age combined with the amount of time lived in the 
country. 

The third parameter that guided development was test design. USCIS made a 
commitment that the redesigned test would be neither more difficult nor easier than the 
current test; but rather, one that was meaningful, standardized, and fair. For example, the 
test should take the same length of time as the current test to administer and require the 
same number of questions according to current USCIS guidance. To this end, USCIS 
developed a process to develop a redesigned test that fell within the specified parameters. 

Test Development 

One of the first tasks undertaken by the USCIS Office of Citizenship was the 
development of a six-phase project plan to guide the redesign initiative. The plan 
included the following phases: Transition, Discovery, Development, Pre-Implementation, 
Implementation, and Post-Implementation. USCIS contracted with the American 
Institutes of Research (AIR) to assist with the Development, Pre-Implementation, 
Implementation, and Post-Implementation phases. Part of the collaboration with AIR 
included the Pilot Study and a Supplemental Study on the civics portion of the redesigned 
test. 

USCIS also utilized the expertise of an adult education expert Technical Advisory Group 
(TAG). The TAG was comprised of eight consultants with expertise in one or more of the 
following areas: adult second language acquisition; test development and administration; 
adult English as a second language instruction and methodology; knowledge of 
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psychometrics in test development; and, an understanding of the naturalization process. 
The TAG provided expert advice regarding language level difficulty of the redesigned 
test items to ensure that the redesigned test coincided with the educational requirements 
for naturalization set forth in Section 312 of the INA. 

Pilot Study 

The target number of civics items for the final test was to remain at 100. In order to 
ensure that we had an over representative number of items in the various topic areas to 
pilot, an additional 42 items were chosen. Once the 142 civics test items were developed, 
USCIS, in collaboration with AIR, conducted a Pilot Study at ten randomly selected 
USCIS District Office sites nationwide. The purpose of the Pilot Study was to determine 
if naturalization applicants who volunteered to participate in the pilot could pass the 
civics portion of the naturalization test using the piloted items. An evaluation of the pilot 
reflected that 92.4% of the volunteer naturalization applicants achieved a passing rate. 

USCIS provided Pilot Study participants with test preparation materials to study for the 
test after they filed for naturalization. These individuals most likely prepared for the 
civics test before participating in the pilot. Results of the Pilot Study analysis are 
classified into the following categories: 

• Low (< 25%) 

• Medium low (25.0 - 49.9%) 

• Medium high (50.0 - 74.9%) 

. High (75.0- 100%) 

The item analysis on the subsequent pages of this document examines how well pilot 
participants scored on the piloted civics items using the categories above. The ranges in 
the chart show whether the majority of applicants scored well (overall) on the item or 
performed poorly. Findings obtained from the Pilot Study served as an important source 
of information throughout the test redesign process. 

Supplemental Study 

USCIS also conducted a Supplemental Study to determine how well a specific population 
would perform on the individual civics test items. This population consisted of 
individuals enrolled in English as a second language (ESL) classes who possessed low- 
beginning to high-beginning levels of English comprehension. By focusing on this 
population USCIS could ascertain how well naturalization applicants with lower levels of 
English literacy would be able to understand and answer the test questions (see Baseline 
Parameters). 

For the Supplemental Study, and unlike the Pilot Study, participants had indirect access 
to the information. In other words, Supplemental Study participants had access to the 
study materials but may or may not have chosen to study prior to participation. Another 
significant difference between the two studies was that participants in the Supplemental 
Study were not volunteers. In summary, the objectives of the Supplemental Study were to 
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focus on performance levels for each individual test item and to help test developers 
identify whether revisions were needed on a particular item to ensure that it was written 
at the appropriate English language level. 

The range for how well applicants performed on each of the trials is provided in the chart 
under the title “Supplemental Study Results” in the following item analysis pages. The 
Supplemental Study results are classified into the categories below: 

• Low (< 25%) 

• Medium low (25.0 - 49.9%) 

• Medium high (50.0 - 74.9%) 

. High (75.0- 100%) 

The Supplemental Study chart provides the result of the trial and, if applicable, re-trial of 
the item. The ranges indicate whether the majority of applicants scored well overall on 
the item or performed poorly. USCIS used information obtained from the Supplemental 
Study as guidance when developing the question and answer items of the redesigned 
civics test. 

Looking Forward 

USCIS developed a test in the context of U.S. naturalization policy that does not 
specifically mandate a formal testing process. Therefore, rather than a punitive 
assessment, the redesigned test can be understood as a vehicle for a naturalization 
candidate to learn about the principles of American democracy and further identify with 
its civic values. 

The Naturalization Test Redesign Project: Civics Item Selection Analysis was created to 
provide insight into the civics portion of the redesign initiative. The data demonstrate that 
applicants who studied for the test achieved an average pass rate of 92.4% on the Pilot 
Study. To assist applicants and adult education teachers, the USCIS Office of Citizenship 
is in the process of creating study materials to accompany the redesigned civics test 
items. These materials will be developed for use both as self-study tools and in the 
classroom. The objective for all USCIS study materials will be to prepare the applicant 
for the naturalization test and further promote civic learning and integration. 
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Final Item Selection Analysis 



Naturalization applicants will still be required to answer six (6) out of ten (10) civics test 
questions correctly to pass the civics portion of the redesigned test. Following this 
measure, test developers were responsible for narrowing down 142 test items to 100 final 
test items. Items were reviewed according to three major areas. 

The first area, linguistic simplicity, involved reviewing the language difficulty of each 
item to ensure that it was at an appropriate linguistic level. The second area focused on 
evaluating and ensuring that test content was meaningful. The third area of review 
involved meeting test design specifications. For example, USCIS worked with a panel of 
experts to select topics covered in the content items and to ensure the test met the 
desirable specifications for the number of questions in each section. 

Linguistic Simplicity 

When an item was developed, linguistic parameters were applied in accordance with the 
statutory requirements of the INA which provide that applicants be able to read, write, 
speak, and understand words in “ordinary usage in the English language.” To ensure that 
the item was developed at the correct linguistic level, USCIS relied on the expert opinion 
of its TAG which followed specific rules for improving language content. These 
guidelines consisted of using: 

• Simple words and phrases 

• Words found in “ordinary usage in the English language” 

• Simple grammar and sentence structure to avoid longer sentences and 
complexity 

• Simplified language without jeopardizing important concepts 

During the item revision process, the TAG endeavored to eliminate unnecessary words, 
whenever possible. 

Meaningful Content 

During the redesign process, some new items were added to expand meaningful content 
and/or add depth of meaning to the test. For example, instead of asking whose rights are 
guaranteed by the Constitution and the Bill of Rights, the revised test item asks the 
applicant to identify two specific individual rights listed in these documents. Additional 
test items were added to extend civic, government, and/or history content. 

Test Design Specifications 

USCIS conducted a literature review of textbooks, developed a matrix, and had a panel of 
experts review the matrix to ensure pertinent parts of American history were covered in 
the redesigned test. The test questions fall within the following sections: Principles of 
American Democracy, System of Government, Rights and Responsibilities, Colonial 
Period and Independence, 1800s, Recent American History and Other Important 
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Historical Information, Geography, Symbols, and Holidays. Some test items were deleted 
due to a limitation on the number of questions included within each section (refer to 
number three in the coding system below). Due to test design limitations, the developers 
were restricted in the amount of questions that could be included. For example, during the 
redesign process, several questions were developed on the concept of “veto.” It was 
determined that, due to test design limitations, there could only be one question on the 
concept of “veto.” In a case such as this, USCIS will develop additional learning 
materials for applicants that will provide a deeper understanding of important concepts 
covered in the final 100 civics test items. 

Coding System 

To guide test developers in the item revision and selection process, a coding system was 
developed and applied to each item. This coding system reflects an analysis of revisions 
to the original item (current naturalization test question) that led to the piloted item and/or 
revisions to the piloted item that led to the final test item. Codes may also indicate an 
analysis of items that should be included in the piloted or final items. The following five 
codes were used: 

1. Added Meaning 

The item was added or a current item was revised in order to add depth of 
meaning. Examples of adding depth of meaning can include: 

• expanding the breadth of a content area, e.g., including a question about a 
time period of history excluded in the current test, 

• deepening the meaning of a current item, e.g., making an item more thought 
provoking, 

• adding an item that reinforces other items, e.g., adding a question about 
geography that would help an applicant better grasp an item on history. 

2. Linguistic Simplicity 

The item was re-written in order to use more ordinary English terms (simple 
words and phrases). The TAG analyzed each item on the test. Where possible, the 
TAG substituted simple words in place of higher level words. They also replaced 
difficult sentence constructions that used the passive voice as well as complex and 
compound sentences with active and simple sentences. Finally, sentence order 
was changed to move more pertinent information toward the end of the sentence. 

3. Eliminated Content 

Redundant content was eliminated in order to expand the depth and breadth of the 
test and to reduce confusion. In addition, certain items were eliminated if they 
required extensive knowledge of a subject area in order to fully grasp the meaning 
of an item. 



8 




Naturalization Test Redesign Project 
Civics Item Selection Analysis 



4. Revised - Due to Test Design 

One or more variables contributed to the item not performing the way it was 
intended. Because the content was considered “fundamental,'’ the item was 
revised. 

5. Deleted - Due to Test Design 

One or more variables contributed to the item not performing the way it was 
intended. If revisions were also unsuccessful, or if the content was represented in 
one or more other items, the item was deleted from the test. Some items were also 
deleted due to the limit of 100 questions that could be included in the final test. 

Analysis 

The subsequent pages of this document describe the revision and selection process for 
each of the 142 piloted civics test items including one item that was not formally piloted, 
but was incorporated into the final test. The item, “What ocean is on the East Coast of the 
United States?” is similar to the piloted item, “What ocean is on the West Coast of the 
United States?” The former item was not piloted due to the similar grammatical structure 
and vocabulary of the latter piloted item. Within this document, the first part of the 
analysis presents each of the final 100 selected civics items for the redesigned 
naturalization test by section (Principles of American Government, System of 
Government, Rights and Responsibilities, Colonial Period and Independence, 1800s, 
Recent American History and Other Important Historical Information, Geography, 
Symbols, and Holidays). The analysis for each of the final 100 selected civics items 
includes: Pilot Study results, Supplemental Study results, and rationale for change. The 
second part of the analysis presents the items that were deleted. The analysis for each of 
the deleted items includes: Pilot Study results, Supplemental Study results, and rationale 
for deleting the piloted items. For a complete list of the 142 pilot test items, see Appendix 
A. For a complete list of the final 100 civics items for the redesigned naturalization test, 
see Appendix B. For a content comparison between the civics items included on the 
current naturalization test and the civics items selected for the redesigned test, see 
Appendix C. 
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A: Principles of American Democracy 

FINAL ITEM: What is the supreme law of the land? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the supreme law of the land? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the supreme law of the land? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What is the supreme law of the United States? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: What does the Constitution do? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What does the Constitution do? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What does the Constitution do? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What is the Constitution? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: The idea of self-government is in the first three words of the 
Constitution. What are these words? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What does “We the People” mean in the Constitution? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What does “We the People” mean in the Constitution? 



Re-trial The idea of self-government is in the first three words of the Constitution. 
Item: What are these words? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 






v' 







RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What is the introduction to the Constitution called? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: What is an amendment? 

PILOT STUDY RESULTS: 



Trial 

Item: 



What is an amendment? 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is an amendment? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Can the Constitution be changed? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: What do we call the first ten amendments to the Constitution? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What do we call the first ten amendments to the Constitution? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What do we call the first ten amendments to the Constitution? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: What are the first 10 amendments to the Constitution called? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: What is one right or freedom from the First Amendment? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one right or freedom from the First Amendment. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one right or freedom from the First Amendment. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Name one right or freedom guaranteed by the first amendment. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: How many amendments does the Constitution have? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



How many amendments does the Constitution have? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



How many amendments does the Constitution have? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: How many changes, or amendments, are there to the Constitution? 



16 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: What did the Declaration of Independence do? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What did the Declaration of Independence do? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What did the Declaration of Independence do? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: What are some of the basic beliefs of the Declaration of Independence? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: What are two rights in the Declaration of Independence? 

PILOT STUDY RESULTS: 



Trial Name two of the natural, or inalienable, rights in the Declaration of 

Item: Independence. 



Pass Rate 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial Name two of the natural, or inalienable, rights in the Declaration of 

Item: Independence. 

Re-trial Inalienable rights are natural rights that people are born with. What are two 
Item: of these rights in the Declaration of Independence? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What are some of the basic beliefs of the Declaration of Independence? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: What is freedom of religion? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What does freedom of religion mean? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What does freedom of religion mean? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 
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A: Principles of American Democracy 



FINAL ITEM: What is the economic system in the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What type of economic system does the U.S. have? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What type of economic system does the U.S. have? 



Re-trial 

Item: 



What is the economic system in the U.S.? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



A: Principles of American Democracy 



FINAL ITEM: What is the “rule of law”? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the “rule of law”? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the “rule of law”? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



21 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 

FINAL ITEM: Name one branch or part of the government. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one branch or part of the government. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one branch or part of the government. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Items: What are the three branches of our government? 
How many branches are there in the United States government? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: What stops one branch of government from becoming too powerful? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one example of checks and balances. 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one example of checks and balances. 



Re-trial What system of government stops one branch of government from becoming 
Item: too powerful? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Who is in charge of the executive branch? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is in charge of the executive branch? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is in charge of the executive branch? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who is the head of the Executive Branch of the U.S. Government? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Who makes federal laws? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who makes federal laws? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who makes federal laws? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who makes the Federal laws in the United States? 



25 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: What are the two parts of the U.S. Congress? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What are the two parts of the United States Congress? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What are the two parts of the United States Congress? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: What makes up Congress? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: How many U.S. Senators are there? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



How many United States Senators are there? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



How many United States Senators are there? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: How many Senators are there in Congress? 
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Coding System 

1 - Added meaning 3 - Eliminated content 5 - Deleted - test design 

2 - Linguistic simplicity 4 - Revised - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: We elect a U.S. Senator for how many years? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



We elect a U.S. Senator for how many years? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



We elect a U.S. Senator for how many years? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: For how long do we elect each Senator? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Who is one of your state’s U.S. Senators? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name your state’s two U.S. Senators. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name your state’s two U.S. Senators. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Name two Senators from your state. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: The House of Representatives has how many voting members? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



The House of Representatives has how many voting members? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



The House of Representatives has how many voting members? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: How many voting members are in the House of Representatives? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: We elect a U.S. Representative for how many years? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



We elect a U.S. Representative for how many years? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



We elect a U.S. Representative for how many years? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: For how long do we elect each member of the House of Representatives? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Name your U.S. Representative. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name your U.S. Representative. 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name your U.S. Representative. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Who does a U.S. Senator represent? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who does a U.S. Senator represent? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who does a U.S. Senator represent? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Why do some states have more Representatives than other states? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



How is each state’s number of Representatives decided? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



How is each state’s number of Representatives decided? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 



34 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: We elect a President for how many years? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



We elect a President for how many years? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



We elect a President for how many years? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: For how long is the President elected? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: In what month do we vote for President? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



In what month are general presidential elections held in the United States? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



In what month are general presidential elections held in the United States? 



Re-trial 

Item: 



When do the people vote for President? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: In what month do we vote for the President? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: What is the name of the President of the United States now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the name of the President of the United States? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the name of the President of the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who is the President of the United States today? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: What is the name of the Vice President of the United States now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the name of the Vice President of the United States? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the name of the Vice President of the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 










V 



RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who is the Vice President of the United States today? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: If the President can no longer serve, who becomes President? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



If the President can no longer serve, who becomes President? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



If the President can no longer serve, who becomes President? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Who becomes President if the President dies? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: If both the President and the Vice President can no longer serve, who 
becomes President? 

PILOT STUDY RESULTS: 



Trial Who becomes President if both the President and the Vice President can no 
Item: longer serve? 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial Who becomes President if both the President and the Vice President can no 
Item: longer serve? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 










V 



RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Who becomes President if both the President and Vice President die? 



40 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Who is the Commander in Chief of the military? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the Commander-in-Chief of the military? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the Commander-in-Chief of the military? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who is Commander-in-Chief of the United States military? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Who signs bills to become laws? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who signs bills to become laws? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who signs bills to become laws? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who signs bills into law? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Who vetoes bills? 

PILOT STUDY RESULTS: 



Trial 

Item: 



Who vetoes bills? 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who vetoes bills? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: What does the President’s Cabinet do? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What does the President’s Cabinet do? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What does the President’s Cabinet do? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What special group advises the President? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: What are two Cabinet-level positions? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name two Cabinet-level positions. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name two Cabinet-level positions. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: What does the judicial branch do? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What does the judicial branch do? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What does the judicial branch do? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What are the duties of the Supreme Court? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: What is the highest court in the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the highest court in the U.S.? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the highest court in the U.S.? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: No change 

Original Item: What is the highest court in the United States? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: How many justices are on the Supreme Court? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



How many justices are on the Supreme Court? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



How many justices are on the Supreme Court? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: How many Supreme Court Justices are there? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Who is the Chief Justice of the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the Chief Justice of the United States? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the Chief Justice of the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who is Chief Justice of the Supreme Court? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Under our Constitution, some powers belong to the federal government. 
What is one power of the federal government? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one thing only the federal government can do. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one thing only the federal government can do. 



Re-trial Our Constitution limits powers to the Federal government. What is one 
Item: power of the Federal government? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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Naturalization Test Redesign Project 
Civics Item Selection Analysis 



B: System of Government 



FINAL ITEM: Under our Constitution, some powers belong to the states. What is one 
power of the states? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is one thing a state government can do? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is one thing a state government can do? 



Re-trial Under our Constitution, powers belong to the states. What is one power of 
Item: the states? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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B: System of Government 



FINAL ITEM: Who is the Governor of your state? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the Governor of your state? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the Governor of your state? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who is the current Governor of the state you live in? 
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B: System of Government 



FINAL ITEM: What is the capital of your state? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the capital (or capital city) of your state? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the capital (or capital city) of your state? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: What is the capital of the state you live in? 
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B: System of Government 



FINAL ITEM: What are the two major political parties in the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What are the two major political parties in the U.S. today? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What are the two major political parties in the U.S. today? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: What are the two major political parties in the United States today? 
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B: System of Government 



FINAL ITEM: What is the political party of the President now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the political party of the President now? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the political party of the President now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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B: System of Government 



FINAL ITEM: What is the name of the Speaker of the House of Representatives now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the Speaker of the House of Representatives now? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the Speaker of the House of Representatives now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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Civics Item Selection Analysis 



C: Rights and Responsibilities 

FINAL ITEM: There are four amendments to the Constitution about who can vote. 
Describe one of them. 

PILOT STUDY RESULTS: 



Trial There are four amendments to the Constitution about who can vote. Describe 

Item: one of them. 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial There are four amendments to the Constitution about who can vote. Describe 

Item: one of them. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Name the amendments that guarantee or address voting rights. 
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Naturalization Test Redesign Project 
Civics Item Selection Analysis 



C: Rights and Responsibilities 



FINAL ITEM: What is one responsibility that is only for United States citizens? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one responsibility that is only for United States citizens. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one responsibility that is only for United States citizens. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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C: Rights and Responsibilities 



FINAL ITEM: What are two rights only for United States citizens? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name two rights that are only for United States citizens. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name two rights that are only for United States citizens. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What is the most important right granted to United States citizens? 
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Naturalization Test Redesign Project 
Civics Item Selection Analysis 



C: Rights and Responsibilities 



FINAL ITEM: What are two rights of everyone living in the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name two rights of everyone living in the U.S. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name two rights of everyone living in the U.S. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Whose rights are guaranteed by the Constitution and the Bill of Rights? 
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Civics Item Selection Analysis 



C: Rights and Responsibilities 



FINAL ITEM: What do we show loyalty to when we say the Pledge of Allegiance? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the Pledge of Allegiance? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the Pledge of Allegiance? 



Re-trial 

Item: 



Why do we say the Pledge of Allegiance? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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Civics Item Selection Analysis 



C: Rights and Responsibilities 



FINAL ITEM: What is one promise you make when you become a United States 
citizen? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one promise you make when you say the Oath of Allegiance. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one promise you make when you say the Oath of Allegiance. 



Re-trial 

Item: 



When you become a citizen, what is one promise you make? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




S 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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C: Rights and Responsibilities 



FINAL ITEM: How old do citizens have to be to vote for President? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who can vote in the U.S. presidential elections? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who can vote in the U.S. presidential elections? 



Re-trial 

Item: 



How old do citizens have to be to vote for President? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: What is the minimum voting age in the United States? 
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Civics Item Selection Analysis 



C: Rights and Responsibilities 



FINAL ITEM: What are two ways that Americans can participate in their democracy? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name two ways that Americans can participate in their democracy. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name two ways that Americans can participate in their democracy. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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Naturalization Test Redesign Project 
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C: Rights and Responsibilities 



FINAL ITEM: When is the last day you can send in federal income tax forms? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



When is the last day you can send in federal income tax forms? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



When is the last day you can send in federal income tax forms? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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C: Rights and Responsibilities 



FINAL ITEM: When must all men register for the Selective Service? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



When must all males register for the Selective Service? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



When must all males register for the Selective Service? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 










V 



RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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D: Colonial Period and Independence 

FINAL ITEM: What is one reason colonists came to America? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one reason why the colonists came to America. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one reason why the colonists came to America. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Why did the Pilgrims come to America? 
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D: Colonial Period and Independence 



FINAL ITEM: Who lived in America before the Europeans arrived? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who lived in America before the Europeans arrived? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who lived in America before the Europeans arrived? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Who helped the Pilgrims in America? 
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D: Colonial Period and Independence 



FINAL ITEM: What group of people was taken to America and sold as slaves? 

PILOT STUDY RESULTS: 



Trial What group of people was taken to America and sold as slaves? 

Item: 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What group of people was taken to America and sold as slaves? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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D: Colonial Period and Independence 



FINAL ITEM: Why did the colonists fight the British? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Why did the colonists fight the British? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Why did the colonists fight the British? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What country did we fight during the Revolutionary War? 
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D: Colonial Period and Independence 



FINAL ITEM: Who wrote the Declaration of Independence? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who wrote the Declaration of Independence? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who wrote the Declaration of Independence? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Who was the main writer of the Declaration of Independence? 
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D: Colonial Period and Independence 



FINAL ITEM: When was the Declaration of Independence adopted? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



When was the Declaration of Independence adopted? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



When was the Declaration of Independence adopted? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: No change. 

Original Item: When was the Declaration of Independence adopted? 
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D: Colonial Period and Independence 



FINAL ITEM: There were 13 original states. Name three . 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



There are 13 original states. Name three. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



There are 13 original states. Name three. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2, 4 

Original Item: What were the original 13 states? 
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D: Colonial Period and Independence 



FINAL ITEM: What happened at the Constitutional Convention? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What happened at the Constitutional Convention? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What happened at the Constitutional Convention? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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D: Colonial Period and Independence 



FINAL ITEM: When was the Constitution written? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



When was the Constitution drafted? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



When was the Constitution drafted? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: In what year was the Constitution written? 
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D: Colonial Period and Independence 



FINAL ITEM: The Federalist Papers supported the passage of the U.S. Constitution. 
Name one of the writers. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one of the writers of the Federalist Papers. 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one of the writers of the Federalist Papers. 



Re-trial The Federalist Papers were a group of essays supporting the passage of the 
Item: U.S. Constitution. Name one of the writers. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 






V 







RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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D: Colonial Period and Independence 



FINAL ITEM: What is one thing Benjamin Franklin is famous for? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one thing Benjamin Franklin is famous for. 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one thing Benjamin Franklin is famous for. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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D: Colonial Period and Independence 



FINAL ITEM: Who is the “Father of Our Country”? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is called the “Father of Our Country”? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is called the “Father of Our Country”? 



Re-trial 

Item: 



Who was the first President and the “Father of Our Country”? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Which President is called the “Father of our Country”? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



D: Colonial Period and Independence 



FINAL ITEM: Who was the first President? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who was the first President? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who was the first President? 



Re-trial 

Item: 



Who was the first President and the “Father of Our Country”? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 








V 




Re-trial 








s 





RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: Who was the first President of the United States? 



79 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



E: 1800s 



FINAL ITEM: What territory did the United States buy from France in 1803? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What territory did the United States buy from France in 1803? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What territory did the United States buy from France in 1803? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



80 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



E: 1800s 



FINAL ITEM: Name one war fought by the United States in the 1800s. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one war fought by the United States in the 1800s. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one war fought by the United States in the 1800s. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



81 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



E: 1800s 



FINAL ITEM: Name the U.S. war between the North and the South. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name the U.S. war between the North and the South. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name the U.S. war between the North and the South. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



82 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



E: 1800s 



FINAL ITEM: Name one problem that led to the Civil War. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one problem that led to the Civil War. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one problem that led to the Civil War. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



83 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



E: 1800s 



FINAL ITEM: What was one important thing that Abraham Lincoln did? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one of the things that Abraham Lincoln did. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one of the things that Abraham Lincoln did. 



Re-trial 

Item: 



What is one important thing that Abraham Lincoln did? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 






s 







RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Who was President during the Civil War? 



84 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



E: 1800s 



FINAL ITEM: What did the Emancipation Proclamation do? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What did the Emancipation Proclamation do? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What did the Emancipation Proclamation do? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: No change. 

Original Item: What did the Emancipation Proclamation do? 



85 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



E: 1800s 



FINAL ITEM: What did Susan B. Anthony do? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What did Susan B. Anthony do? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What did Susan B. Anthony do? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



86 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F : Recent American History and Other 
Important Historical Information 

FINAL ITEM: Name one war fought by the United States in the 1900s. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one war fought by the United States in the 1900s. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one war fought by the United States in the 1900s. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



87 

Coding System 

1 - Added meaning 3 - Eliminated content 5 - Deleted - test design 

2 - Linguistic simplicity 4 - Revised - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: Who was President during World War I? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who was President during World War I? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who was President during World War I? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



88 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: Who was President during the Great Depression and World War II? 

PILOT STUDY RESULTS: 



Trial Who was President during the Great Depression and World War II? 

Item: 



Pass Rate 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who was President during the Great Depression and World War II? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



89 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: Who did the United States fight in World War II? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



The United States fought Japan, Germany, and Italy during which war? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



The United States fought Japan, Germany, and Italy during which war? 



Re-trial What countries did the United States fight in World War II? 

Item: 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Name some countries that were our enemies during World War II. 



90 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: Before he was President, Eisenhower was a general. What war was he 
in? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Which U.S. World War II general later became President? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Which U.S. World War II general later became President? 



Re-trial Before becoming President, Eisenhower was a General. What war was he 

Item: in? 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 



91 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: During the Cold War, what was the main concern of the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What was the main concern of the United States during the Cold War? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What was the main concern of the United States during the Cold War? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 



92 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: What movement tried to end racial discrimination? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What movement tried to end racial discrimination? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What movement tried to end racial discrimination? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



93 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: What did Martin Luther King, Jr. do? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What did Martin Luther King, Jr. do? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What did Martin Luther King, Jr. do? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Who was Martin Luther King, Jr.? 



94 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: What major event happened on September 11, 2001 in the United 
States? 

PILOT STUDY RESULTS: 



Trial What major event happened on September 11, 2001, in the United States? 

Item: 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What major event happened on September 11, 2001, in the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



95 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



F: Recent American History and Other 
Important Historical Information 



FINAL ITEM: Name one American Indian tribe in the United States. 

PILOT STUDY RESULTS: 



Trial Name one of the major American Indian tribes in the United States. 

Item: 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one of the major American Indian tribes in the United States. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 



96 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



G: Geography 

FINAL ITEM: Name one of the two longest rivers in the United States. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one of the two longest rivers in the United States. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one of the two longest rivers in the United States. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



97 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



G: Geography 



FINAL ITEM: What ocean is on the West Coast of the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What ocean is on the west coast of the United States? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What ocean is on the west coast of the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 



98 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



G: Geography 



FINAL ITEM: What ocean is on the East Coast of the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 



Not applicable. 



Pass Rate 



Low 


Medium Low 


Medium High 


High 











SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Not applicable. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 










y 


Re-trial 










y 



RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 

Note: Due to the similar grammatical structure and vocabulary between this item and the 
previous one, this item was not piloted. This decision was informed by the advice of the 
TAG. Understanding basic geography is a prerequisite to understanding history. Without 
basic knowledge, it would be impossible for an applicant to fully understand major 
phenomena in U.S. history. 



99 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



G: Geography 



FINAL ITEM: Name one U.S. territory. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one U.S. territory. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one U.S. territory. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



G: Geography 



FINAL ITEM: Name one state that borders Canada. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one state that borders Canada. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one state that borders Canada. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



G: Geography 



FINAL ITEM: Name one state that borders Mexico. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one state that borders on Mexico. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one state that borders on Mexico. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



G: Geography 



FINAL ITEM: What is the capital of the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the capital of the U.S.? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the capital of the U.S.? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



G: Geography 



FINAL ITEM: Where is the Statue of Liberty? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Where is the Statue of Liberty? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Where is the Statue of Liberty? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



H: Symbols 

FINAL ITEM: Why does the flag have 13 stripes? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Why does the flag have 13 stripes? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Why does the flag have 13 stripes? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2 

Original Item: What do the stripes on the flag represent? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



H: Symbols 



FINAL ITEM: Why does the flag have 50 stars? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Why does the flag have 50 stars? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Why does the flag have 50 stars? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: What do the stars on the flag mean? 



106 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



H: Symbols 



FINAL ITEM: What is the name of the national anthem? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the name of the National Anthem? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the name of the National Anthem? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: What is the national anthem of the United States? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



I: Holidays 

FINAL ITEM: When do we celebrate Independence Day? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



When do we celebrate Independence Day? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



When do we celebrate Independence Day? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2 

Original Item: What do we celebrate on the 4 th of July? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



I: Holidays 



FINAL ITEM: Name two national U.S. holidays. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name two national U.S. holidays. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name two national U.S. holidays. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Name one important idea found in the Declaration of Independence. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one important idea found in the Declaration of Independence. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one important idea found in the Declaration of Independence. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2, 3 

Original Item: What are some of the basic beliefs of the Declaration of Independence? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What do we call changes to the Constitution? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What do we call changes to the Constitution? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What do we call changes to the Constitution? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 3 

Original Item: What do we call changes to the Constitution? 



Ill 

Coding System 

1 - Added meaning 3 - Eliminated content 5 - Deleted - test design 

2 - Linguistic simplicity 4 - Revised - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What are the three branches of our government? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What are the three branches or parts of the government? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What are the three branches or parts of the government? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2, 3 

Original Item: What are the three branches of our government? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: How many U.S. Senators does each state have? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



How many U.S. Senators does each state have? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



How many U.S. Senators does each state have? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2, 3 

Original Item: Why are there 100 Senators in the United States Senate? 
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Coding System 

1 - Added meaning 3 - Eliminated content 5 - Deleted - test design 

2 - Linguistic simplicity 4 - Revised - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who does a U.S. Representative represent? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who does a U.S. Representative represent? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who does a U.S. Representative represent? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What decides each state’s number of U.S. Representatives? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What decides each state’s number of U.S. Representatives? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What decides each state’s number of U.S. Representatives? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Why do we have three branches of government? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Why do we have three branches of government? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Why do we have three branches of government? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: How many branches are there in the United States government? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: How old must a President be? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



How old must a President be? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



How old must a President be? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: To become President of the United States, what must the person be 
at birth? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



To become President of the United States, what must the person be at birth? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



To become President of the United States, what must the person be at birth? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2, 3, 5 

Original Item: What are some of the requirements to be eligible to become President? 
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Coding System 

1 - Added meaning 3 - Eliminated content 5 - Deleted - test design 

2 - Linguistic simplicity 4 - Revised - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who is the President now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the President now? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the President now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2, 3 

Original Item: Who is the President of the United States today? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who is the Vice President now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the Vice President now? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the Vice President now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 










V 



RATIONALE FOR CHANGE: 



Implementation Code: 2, 5 

Original Item: Who is the Vice President of the United States today? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: How many full terms can a President serve? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



How many full terms can a President serve? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



How many full terms can a President serve? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 3 

Original Item: How many full terms can a President serve? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What is a veto? 
PILOT STUDY RESULTS: 



Trial 

Item: 



What is a veto? 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is a veto? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What Cabinet-level agency advises the President on foreign policy? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What Cabinet-level agency advises the President on foreign policy? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What Cabinet-level agency advises the President on foreign policy? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who confirms Supreme Court justices? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who confirms Supreme Court justices? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who confirms Supreme Court justices? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 5 

Original Item: Who nominates judges for the Supreme Court? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who nominates justices to the Supreme Court? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who nominates justices to the Supreme Court? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who nominates justices to the Supreme Court? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3 

Original Item: Who nominates judges for the Supreme Court? 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What does it mean that the U.S. Constitution is a constitution of 
limited powers' 1 

PILOT STUDY RESULTS: 



Trial What does it mean that the U.S. Constitution is a constitution of limited 
Item: powers ? 



Pass Rate 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial What does it mean that the U.S. Constitution is a constitution of limited 
Item: powers' 1 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What is the majority political party in the House of Representatives 
now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the majority political party in the House of Representatives now? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the majority political party in the House of Representatives now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What is the political party of the majority in the Senate now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the political party of the majority in the Senate now? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the political party of the majority in the Senate now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who is the Senate Majority Leader now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the Senate Majority Leader now? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the Senate Majority Leader now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who is the Secretary of State now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the Secretary of State now? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the Secretary of State now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who is the Attorney General now? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who is the Attorney General now? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who is the Attorney General now? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Is the current President in his first or second term? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Is the current President in his first or second term? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Is the current President in his first or second term? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What is self-government? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is self-government? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is self-government? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Who governs the people in a self-governed country? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Who governs the people in a self-governed country? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Who governs the people in a self-governed country? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: What are “inalienable rights”? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What are “inalienable rights”? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What are “inalienable rights”? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Where did most of America’s colonists come from before the 
Revolution? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Where did most of America’s colonists come from before the Revolution? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Where did most of America’s colonists come from before the Revolution? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 
Implementation Code: 1, 3 

Original Item: Not applicable. 
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1 - Added meaning 

2 - Linguistic simplicity 



Coding System 
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Deleted Items 



DELETED ITEM: Why were the colonists upset with the British government? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Why were the colonists upset with the British government? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Why were the colonists upset with the British government? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: What group of essays supported passage of the U.S. Constitution? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What group of essays supported passage of the U.S. Constitution? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What group of essays supported passage of the U.S. Constitution? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: What country sold the Louisiana Territory to the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What country sold the Louisiana Territory to the United States? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What country sold the Louisiana Territory to the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: In 1803, the United States bought a large amount of land from 
France. Where was that land? 

PILOT STUDY RESULTS: 



Trial In 1803, the United States bought a large amount of land from France. 
Item: Where was that land? 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial In 1803, the United States bought a large amount of land from France. 
Item: Where was that land? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: What did the abolitionists try to end before the Civil War? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What did the abolitionists try to end before the Civil War? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What did the abolitionists try to end before the Civil War? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 5 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: What international organization was established after World War II 
(WWII) to keep the world at peace? 

PILOT STUDY RESULTS: 



Trial What international organization was established after World War II (WWII) 
Item: to keep the world at peace? 



Pass Rate 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial What international organization was established after World War II (WWII) 
Item: to keep the world at peace? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 5 

Original Item: Name one of the purposes of the United Nations. 
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Deleted Items 



DELETED ITEM: What alliance of North American and European countries was 
created during the Cold War? 

PILOT STUDY RESULTS: 



Trial What alliance of North American and European countries was created 
Item: during the Cold War? 



Pass Rate 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial What alliance of North American and European countries was created 
Item: during the Cold War? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 5 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: Martin Luther King. Jr. had a dream for America. What was his 
dream? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Martin Luther King. Jr. had a dream for America. What was his dream? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Martin Luther King. Jr. had a dream for America. What was his dream? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 2, 3 

Original Item: Who was Martin Luther King, Jr.? 
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Deleted Items 



DELETED ITEM: What country is on the northern border of the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What country is on the northern border of the United States? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What country is on the northern border of the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: Where is the Grand Canyon? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Where is the Grand Canyon? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Where is the Grand Canyon? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: What country is on the southern border of the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What country is on the southern border of the United States? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What country is on the southern border of the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 



147 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Name one large mountain range in the United States. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name one large mountain range in the United States. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name one large mountain range in the United States. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: What is the tallest mountain in the United States? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



What is the tallest mountain in the United States? 



Low 


Medium Low 


Medium High 


High 






V 





SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



What is the tallest mountain in the United States? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 5 

Original Item: Not applicable. 



149 



1 - Added meaning 

2 - Linguistic simplicity 



Coding System 

3 - Eliminated content 

4 - Revised - test design 



5 - Deleted - test design 








Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Deleted Items 



DELETED ITEM: Name the state that is in the middle of the Pacific Ocean. 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Name the state that is in the middle of the Pacific Ocean. 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Name the state that is in the middle of the Pacific Ocean. 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 




V 








Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 1, 3, 5 

Original Item: Not applicable. 
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Deleted Items 



DELETED ITEM: Why do we have 13 stripes on the flag? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



Why do we have 13 stripes on the flag? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



Why do we have 13 stripes on the flag? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 






V 






Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2, 3 

Original Item: What do the stripes on the flag represent? 
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Deleted Items 



DELETED ITEM: On the Fourth of July, we celebrate independence from what 
country? 

PILOT STUDY RESULTS: 



Trial 

Item: 

Pass Rate 



On the Fourth of July, we celebrate independence from what country? 



Low 


Medium Low 


Medium High 


High 








V 



SUPPLEMENTAL STUDY RESULTS: 



Trial 

Item: 



On the Fourth of July, we celebrate independence from what country? 



Re-trial 

Item: 



Not applicable. 



Pass Rate 


Low 


Medium 

Low 


Medium 

High 


High 


N/A 


Trial 












Re-trial 













RATIONALE FOR CHANGE: 



Implementation Code: 2, 3, 5 

Original Item: Independence Day celebrates independence from whom? 
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Appendix A 

Civics (History and Government) Items for the Pilot Naturalization Test 

AMERICAN GOVERNMENT 
A: Principles of American Democracy 

1. Name one important idea found in the Declaration of Independence. 

2. What is the supreme law of the land? 

3. What does the Constitution do? 

4. What does “We the People” mean in the Constitution? 

5. What do we call changes to the Constitution? 

6. What is an amendment? 

7. What do we call the first ten amendments to the Constitution? 

8. Name one right or freedom from the First Amendment. 

9. How many amendments does the Constitution have? 

10. What did the Declaration of Independence do? 

11. What does freedom of religion mean? 

12. What type of economic system does the U.S. have? 

B: System of Government 

13. What are the three branches or parts of the government? 

14. Name one branch or part of the government. 

15. Who is in charge of the executive branch? 

16. Who makes federal laws? 

17. What are the two parts of the United States Congress? 

18. How many United States Senators are there? 

19. We elect a U.S. Senator for how many years? 
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20. Name your state’s two U.S. Senators. 

21. How many U.S. Senators does each state have? 

22. The House of Representatives has how many voting members? 

23. We elect a U.S. Representative for how many years? 

24. Name your U.S. Representative. 

25. Who does a U.S. Senator represent? 

26. Who does a U.S. Representative represent? 

27. What decides each state’s number of U.S. Representatives? 

28. How is each state’s number of Representatives decided? 

29. Why do we have three branches of government? 

30. Name one example of checks and balances. 

31. We elect a President for how many years? 

32. How old must a President be? 

33. To become President of the United States, what must the person be at birth? 

34. Who is the President now? 

35. What is the name of the President of the United States? 

36. Who is the Vice President now? 

37. What is the name of the Vice President of the United States? 

38. If the President can no longer serve, who becomes President? 

39. Who becomes President if both the President and the Vice President can no longer 
serve? 

40. Who is the Commander-in-Chief of the military? 

41. How many full terms can a President serve? 
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42. Who signs bills to become laws? 

43. Who vetoes bills? 

44. What is a veto? 

45. What does the President’s Cabinet do? 

46. Name two Cabinet-level positions. 

47. What Cabinet-level agency advises the President on foreign policy? 

48. What does the judicial branch do? 

49. Who confirms Supreme Court justices? 

50. Who is the Chief Justice of the United States? 

51. How many justices are on the Supreme Court? 

52. Who nominates justices to the Supreme Court? 

53. Name one thing only the federal government can do. 

54. What is one thing a state government can do? 

55. What does it mean that the U.S. Constitution is a constitution of limited powersl 

56. Who is the Governor of your state? 

57. What is the capital (or capital city) of your state? 

58. What are the two major political parties in the U.S. today? 

59. What is the highest court in the U.S.? 

60. What is the majority political party in the House of Representatives now? 

61. What is the political party of the majority in the Senate now? 

62. What is the political party of the President now? 

63. Who is the Speaker of the House of Representatives now? 

64. Who is the Senate Majority Leader now? 
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65. In what month are general presidential elections held in the United States? 

66. When must all males register for the Selective Service? 

67. Who is the Secretary of State now? 

68. Who is the Attorney General now? 

69. Is the current President in his first or second term? 

C: Rule of Law 

70. What is self-government? 

71. Who governs the people in a self-governed country? 

72. What is the “rule of law”? 

73. What are “inalienable rights”? 

D: Rights and Responsibilities 

74. There are four amendments to the Constitution about who can vote. Describe one of 
them. 

75. Name one responsibility that is only for United States citizens. 

76. Name two rights that are only for United States citizens. 

77. Name two rights of everyone living in the U.S. 

78. What is the Pledge of Allegiance? 

79. Name one promise you make when you say the Oath of Allegiance. 

80. Who can vote in the U.S. presidential elections? 

81. Name two ways that Americans can participate in their democracy. 

82. When is the last day you can send in federal income tax forms? 

83. Name two of the natural, or inalienable, rights in the Declaration of Independence. 
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AMERICAN HISTORY 
A: Colonial and Independence 

84. Who wrote the Declaration of Independence? 

85. When was the Declaration of Independence adopted? 

86. Name one reason why the colonists came to America. 

87. What happened at the Constitutional Convention? 

88. Why did the colonists fight the British? 

89. When was the Constitution drafted? 

90. There are 13 original states. Name three. 

91. What group of people was taken to America and sold as slaves? 

92. Who lived in America before the Europeans arrived? 

93. Where did most of America’s colonists come from before the Revolution? 

94. Why were the colonists upset with the British government? 

95. Name one thing Benjamin Franklin is famous for. 

96. Who is called the “Father of Our Country”? 

97. Who was the first President? 

98. Name one of the writers of the Federalist Papers. 

99. What group of essays supported passage of the U.S. Constitution? 

B: 1800s 

100. Name one war fought by the United States in the 1800s. 

101. What territory did the United States buy from France in 1803? 

102. What country sold the Fouisiana Territory to the United States? 

103. In 1803, the United States bought a large amount of land from France. Where was 
that land? 
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104. Name one of the things that Abraham Lincoln did. 

105. Name the U.S. war between the North and the South. 

106. Name one problem that led to the Civil War. 

107. What did the Emancipation Proclamation do? 

108. What did the abolitionists try to end before the Civil War? 

109. What did Susan B. Anthony do? 

C: Recent American History and Other Important 
Historical Information 

1 10. Name one war fought by the United States in the 1900s. 

111. Who was President during World War I? 

112. The United States fought Japan, Germany, and Italy during which war? 

113. What was the main concern of the United States during the Cold War? 

1 14. What major event happened on September 11, 2001, in the United States? 

1 15. What international organization was established after World War II (WWII) to keep 
the world at peace? 

116. What alliance of North America and European countries was created during the 
Cold War? 

1 17. Who was President during the Great Depression and World War II? 

118. Which U.S. World War II general later became President? 

119. What did Martin Luther King, Jr. do? 

120. Martin Luther King, Jr. had a dream for America. What was his dream? 

121. What movement tried to end racial discrimination? 

122. Name one of the major American Indian tribes in the United States. 
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INTEGRATED CIVICS 
A: Geography 

123. Name one of the two longest rivers in the United States. 

124. What ocean is on the west coast of the United States? 

125. What country is on the northern border of the United States? 

126. Where is the Grand Canyon? 

127. Where is the Statue of Liberty? 

128. What country is on the southern border of the United States? 

129. Name one large mountain range in the United States. 

130. What is the tallest mountain in the United States? 

131. Name one U.S. territory. 

132. Name the state that is in the middle of the Pacific Ocean. 

133. Name one state that borders Canada. 

134. Name one state that borders on Mexico. 

135. What is the capital of the U.S.? 

B: Symbols 

136. Why does the flag have 13 stripes? 

137. Why do we have 13 stripes on the flag? 

138. Why does the flag have 50 stars? 

139. What is the name of the National Anthem? 



159 




Naturalization Test Redesign Project 
Civics Item Selection Analysis 

C: Holidays 

140. On the Fourth of July we celebrate independence from what country? 

141. When do we celebrate Independence Day? 

142. Name two national U.S. holidays. 
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Appendix B 

Final Civics (History and Government) Items for the 
Redesigned Naturalization Test 

AMERICAN GOVERNMENT 

A: Principles of American Democracy 

1. What is the supreme law of the land? 

2. What does the Constitution do? 

3. The idea of self-government is in the first three words of the Constitution. What are 
these words? 

4. What is an amendment? 

5. What do we call the first ten amendments to the Constitution? 

6. What is one right or freedom from the First Amendment? 

7. How many amendments does the Constitution have? 

8. What did the Declaration of Independence do? 

9. What are two rights in the Declaration of Independence? 

10. What is freedom of religion? 

11. What is the economic system in the United States? 

12. What is the “rule of law”? 

B: System of Government 

13. Name one branch or part of the government. 

14. What stops one branch of government from becoming too powerful? 

15. Who is in charge of the executive branch? 

16. Who makes federal laws? 

17. What are the two parts of the U.S. Congress? 

18. How many U.S. Senators are there? 
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19. We elect a U.S. Senator for how many years? 

20. Who is one of your state’s U.S. Senators? 

21. The House of Representatives has how many voting members? 

22. We elect a U.S. Representative for how many years? 

23. Name your U.S. Representative. 

24. Who does a U.S. Senator represent? 

25. Why do some states have more Representatives than other states? 

26. We elect a President for how many years? 

27. In what month do we vote for President? 

28. What is the name of the President of the United States now? 

29. What is the name of the Vice President of the United States now? 

30. If the President can no longer serve, who becomes President? 

31. If both the President and the Vice President can no longer serve, who becomes 
President? 

32. Who is the Commander in Chief of the military? 

33. Who signs bills to become laws? 

34. Who vetoes bills? 

35. What does the President’s Cabinet do? 

36. What are two Cabinet-level positions? 

37. What does the judicial branch do? 

38. What is the highest court in the United States? 

39. How many justices are on the Supreme Court? 

40. Who is the Chief Justice of the United States? 
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41. Under our Constitution, some powers belong to the federal government. What is one 
power of the federal government? 

42. Under our Constitution, some powers belong to the states. What is one power of the 
states? 

43. Who is the Governor of your state? 

44. What is the capital of your state? 

45. What are the two major political parties in the United States? 

46. What is the political party of the President now? 

47. What is the name of the Speaker of the House of Representatives now? 

C: Rights and Responsibilities 

48. There are four amendments to the Constitution about who can vote. Describe one of 
them. 

49. What is one responsibility that is only for United States citizens? 

50. What are two rights only for United States citizens? 

51. What are two rights of everyone living in the United States? 

52. What do we show loyalty to when we say the Pledge of Allegiance? 

53. What is one promise you make when you become a United States citizen? 

54. How old do citizens have to be to vote for President? 

55. What are two ways that Americans can participate in their democracy? 

56. When is the last day you can send in federal income tax forms? 

57. When must all men register for the Selective Service? 
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AMERICAN HISTORY 
A: Colonial Period and Independence 

58. What is one reason colonists came to America? 

59. Who lived in America before the Europeans arrived? 

60. What group of people was taken to America and sold as slaves? 

61. Why did the colonists fight the British? 

62. Who wrote the Declaration of Independence? 

63. When was the Declaration of Independence adopted? 

64. There were 13 original states. Name three . 

65. What happened at the Constitutional Convention? 

66. When was the Constitution written? 

67. The Federalist Papers supported the passage of the U.S. Constitution. Name one of 
the writers. 

68. What is one thing Benjamin Franklin is famous for? 

69. Who is the “Father of Our Country”? 

70. Who was the first President? 

B: 1800s 

71. What territory did the United States buy from France in 1803? 

72. Name one war fought by the United States in the 1800s. 

73. Name the U.S. war between the North and the South. 

74. Name one problem that led to the Civil War. 

75. What was one important thing that Abraham Fincoln did? 

76. What did the Emancipation Proclamation do? 

77. What did Susan B. Anthony do? 
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C: Recent American History and Other Important 
Historical Information 

78. Name one war fought by the United States in the 1900s. 

79. Who was President during World War I? 

80. Who was President during the Great Depression and World War II? 

81. Who did the United States fight in World War II? 

82. Before he was President, Eisenhower was a general. What war was he in? 

83. During the Cold War, what was the main concern of the United States? 

84. What movement tried to end racial discrimination? 

85. What did Martin Luther King, Jr. do? 

86. What major event happened on September 11, 2001 in the United States? 

87. Name one American Indian tribe in the United States. 

INTEGRATED CIVICS 
A: Geography 

88. Name one of the two longest rivers in the United States. 

89. What ocean is on the West Coast of the United States? 

90. What ocean is on the East Coast of the United States? 

91. Name one U.S. territory. 

92. Name one state that borders Canada. 

93. Name one state that borders Mexico. 

94. What is the capital of the United States? 

95. Where is the Statue of Liberty? 
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B: Symbols 

96. Why does the flag have 13 stripes? 

97. Why does the flag have 50 stars? 

98. What is the name of the national anthem? 

C: Holidays 

99. When do we celebrate Independence Day? 

100. Name two national U.S. holidays. 
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Note 

The reading and writing items have been redacted from this report. Although the vocabulary that 
is contained within individual test items will be released to the public, actual reading and writing 
test items will not. 
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Table 2.1 : USCIS District Offices Sampled, with Key Characteristics 



Number 


Office 


Region 


Average number of 
applications processed 
per year 


Average number of 
applications processed 
per month 


1 


Albany 


Northeast 


1,547 


129 


2 


Boston 


Northeast 


15,683 


1,307 


3 


Kansas City 


Midwest 


3,540 


295 


4 


Charleston 


South 


146 


12 


5 


El Paso 


South 


2,732 


228 


6 


San Antonio 


South 


6,752 


563 


7 


Miami 


South 


30,413 


2,534 


8 


Denver 


West 


6,198 


516 


9 


Tucson 


West 


1,878 


156 


10 


Yakima 


West 


302 


25 


Total 


69,191 


5,765 
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Table 2.2: Age of Current Applicants, by Participation Status 



Age 


Participated 


Did not participate 




N 


Percentage* 


N 


Percentage ** 


1 8-24 years 


1,018 


15.5 


3,831 


11.0 


25-39 years 


3,080 


47.0 


13,085 


37.6 


40-49 years 


1,465 


22.4 


6,942 


20.0 


50-64 years 


804 


12.3 


6,653 


19.1 


65+ years 


187 


2.9 


4,278 


12.3 


Total 


6,554 


100 


34,789 


100 



* excludes 218 cases for which A-numbers were missing 
** excludes 452 cases for which A-numbers were missing 
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Table 2.3: Gender of Current Applicants, by Participation Status 



Gender 


Participated 


Did not participate 




N 


Percentage* 


N 


Percentage ** 


Female 


3,500 


53.4 


20,000 


57.5 


Male 


3,054 


46.6 


14,785 


42.5 


Total 


6,554 


100 


34,785 


100 



* excludes 218 cases for which A-numbers were missing 
** excludes 456 cases for which A-numbers were missing 
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List of Supporting Tables 



Table 2.4: Race of Current Applicants, by Participation Status 



Race 


Participated 


Did Not Participate 




N 


Percentage* 


N 


Percentage ** 


White 


3,801 


59.0 


22,005 


64.7 


Black 


885 


13.7 


4,947 


14.5 


Asian 


1,298 


20.1 


5,001 


14.7 


American Indian/Alaska Native 


9 


.1 


57 


.2 


Other 


456 


7.1 


2,021 


5.9 


Total 


6,449 


100 


34,031 


100 



* excludes 323 cases for which A-numbers were missing 
** excludes 1,210 cases for which A-numbers were missing 
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Table 2.5: Geographic Region of Birth of Current Applicants, by Participation 
Status 



Region 


Participated 


Did not participate 




N 


Percentage * 


N 


Percentage ** 


North America 


167 


2.6 


298 


.9 


Central America 


1,587 


24.7 


8,927 


26.2 


South America 


864 


13.5 


4,261 


12.5 


Caribbean 


1,116 


17.4 


11,051 


32.4 


Europe 


859 


13.4 


2,926 


8.6 


Asia 


1,403 


21.8 


5,131 


15.0 


Africa 


402 


6.3 


1,495 


4.4 


Oceania 


25 


.4 


37 


.1 


Total 


6,423 


100 


34,126 


100 



* excludes 349 cases for which A-numbers were missing 
** excludes 1,115 cases for which A-numbers were missing 
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Table 2.6: Age of Prospective Applicants 



Age 


N 


Percentage* 


1 8-24 years 


89 


7.8 


25-39 years 


375 


32.8 


40-49 years 


225 


19.7 


50-64 years 


255 


22.3 


65+ years 


200 


17.5 


Total 


1,144 


100 



* excludes 85 cases for which age information was not provided 
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176 





Naturalization Test Redesign Project 
Civics Item Selection Analysis 



Table 2.7: Gender of Prospective Applicants 



Gender 


N 


Percentage* 


Female 


785 


68.8 


Male 


356 


31.2 


Total 


1,141 


100 



* excludes 88 cases for which gender information was not provided 
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Table 2.8: Region of Birth of Prospective Applicants 



Region 


N 


Percentage* 


North America 


1 


0.1 


Central America 


448 


38.8 


South America 


63 


5.5 


Caribbean 


58 


5.0 


Europe 


69 


6.0 


Asia 


407 


35.2 


Africa 


110 


9.5 


Oceania 


0 


0.0 


Total 


1,156 


100 



* excludes 73 cases for which geographic region information was not provided 
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Table 2.9: Number of Years Prospective Applicants Have Lived in the U.S. 



Years Living in the U.S. 


N 


Percentage 


Fewer than 5 years 


416 


33.9 


5-1 0 years 


465 


37.8 


11-20 years 


261 


21.2 


More than 20 years 


87 


7.1 


Total 


1,229 


100 
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Table 2.10: NRS Level of Prospective Applicants 



NRS Level 


N 


Percentage* 


Beginning 


63 


9.7 


Low Beginning 


69 


10.6 


High Beginning 


120 


18.5 


Low Intermediate 


132 


20.3 


High Intermediate 


119 


18.3 


Advanced 


146 


22.5 


Total 


649 


100 



* excludes 580 cases for which NRS level information was not provided 



180 





Naturalization Test Redesign Project 
Civics Item Selection Analysis 



List of Supporting Tables 



Table 4.1 : Observed Pilot Test Administrations, by Test Site 1 



Test Site 


Number of observed 
interviews 


Number of participants taking 
pilot test 


Albany 


7 


6 


Boston 


9 


6 


Charleston 


4 


2 


Denver 


12 


7 


El Paso 


13 


8 


Kansas City 


12 


5 


Miami 


15 


11 


San Antonio 


7 


7 


Tucson 


8 


7 


Total 


87 


59 



No observations were conducted in Yakima because of a scheduling conflict. 
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Table 4.2: Item Properties of New Civics Test Items for Current Applicants 



Item 

number 


Item text 


N 


Percent 

correct 


Item 

discrimination 


Flag 


1 


Who lived in America before the Europeans 
arrived? 


524 


96.6 


0.24 


* 


2 


Name one of the writers of the Federalist Papers. 


524 


68.1 


0.43 


□ 


3 


Name two rights that are only for United States 
citizens. 


523 


93.9 


0.39 


♦ 


4 


Who is the President now? 


580 


99.5 


0.09 


* 


5 


Who makes federal laws? 


524 


94.1 


0.24 


♦* 


6 


Who signs bills to become laws? 


522 


98.9 


0.27 




7 


What are the two parts of the United States 
Congress? 


521 


87.7 


0.55 


• 


8 


What is the political party of the majority in the 
Senate now? 


519 


88.8 


0.56 


• 


9 


What is the majority political party in the House of 
Representatives now? 


518 


87.6 


0.58 


• 


10 


What do we call changes to the Constitution? 


518 


95.8 


0.35 




11 


What are the three branches or parts of the 
government? 


495 


97.2 


0.38 




12 


Name one important idea found in the Declaration 
of Independence. 


495 


82.2 


0.46 


3S 


13 


Name two Cabinet-level positions. 


495 


80.6 


0.51 


3S 


14 


To become President of the United States, what 
must the person be at birth? 


495 


96.8 


0.42 




15 


Name one war fought by the United States in the 
1900s. 


495 


91.9 


0.44 


♦ 


16 


What did Martin Luther King, Jr. do? 


494 


96.0 


0.32 




17 


Name two rights of everyone living in the U.S. 


492 


89.6 


0.46 


• 


18 


The House of Representatives has how many 
voting members? 


547 


93.1 


0.42 


♦ 


19 


Who is the Vice President now? 


546 


97.8 


0.35 




20 


Is the current President in his first or second term? 


488 


98.0 


0.31 




21 


Name one problem that led to the Civil War. 


491 


84.3 


0.58 


3S 


22 


What are “inalienable rights”? 


491 


69.7 


0.59 


□ 


23 


Who becomes President if both the President and 
the Vice President can no longer serve? 


549 


93.4 


0.37 


♦ 


24 


What is the capital (or capital city) of your state? 


492 


98.8 


0.23 


* 


25 


How many United States Senators are there? 


491 


96.7 


0.35 




26 


What is the name of the National Anthem? 


491 


91.4 


0.36 


♦ 


27 


What does it mean that the U.S. Constitution is a 
constitution of limited powers? 


490 


65.1 


0.60 


□ 


28 


Who was President during the Great Depression 
and World War II? 


489 


74.6 


0.57 


O 


29 


Who is the Secretary of State now? 


489 


82.0 


0.57 


3S 


30 


What is an amendment? 


490 


88.0 


0.51 


• 


31 


What country sold the Louisiana Territory to the 
United States? 


485 


86.6 


0.55 


• 


32 


Name your U.S. Representative. 


484 


65.5 


0.42 


□ 
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Item 

number 


Item text 


N 


Percent 

correct 


Item 

discrimination 


Flag 


33 


Name the U.S. war between the North and the 
South. 


890 


85.7 


0.59 


• 


34 


Who wrote the Declaration of Independence? 


485 


89.7 


0.34 


• 


35 


We elect a President for how many years? 


485 


98.8 


0.16 


* 


36 


In 1803, the United States bought a large amount of 
land from France. Where was that land? 


484 


87.6 


0.59 


• 


37 


Who is the Chief Justice of the United States? 


540 


78.7 


0.32 


O 


38 


Name two of the natural, or inalienable, rights in the 
Declaration of Independence. 


482 


61.8 


0.44 


□ 


39 


Name the state that is in the middle of the Pacific 
Ocean. 


482 


91.5 


0.55 


♦ 


40 


What major event happened on September 1 1 , 
2001, in the United States? 


483 


96.7 


0.33 




41 


Name one reason why the colonists came to 
America? 


480 


95.2 


0.45 




42 


What is the Pledge of Allegiance? 


480 


75.2 


0.53 


o 


43 


How many justices are on the Supreme Court? 


536 


91.6 


0.46 


♦ 


44 


Why do we have 1 3 stripes on the flag? 


536 


99.1 


0.29 




45 


How many amendments does the Constitution 
have? 


480 


94.2 


0.35 


♦ 


46 


Which U.S. World War II general later became 
President? 


480 


65.2 


0.56 


□ 


47 


Who was the first President? 


536 


99.3 


0.15 


* 


48 


What happened at the Constitutional Convention? 


477 


75.9 


0.64 


o 


49 


Name one state that borders on Mexico. 


480 


95.2 


0.46 




50 


How many full terms can a President serve? 


480 


97.3 


0.32 




51 


What movement tried to end racial discrimination? 


464 


76.5 


0.57 


o 


52 


Name one of the major American Indian tribes in 
the United States. 


464 


85.8 


0.55 


• 


53 


What is self-government? 


464 


76.9 


0.64 


o 


54 


When do we celebrate Independence Day? 


464 


99.8 


0.14 


* 


55 


In what month are general presidential elections 
held in the United States? 


464 


98.3 


0.32 




56 


Name one right or freedom from the First 
Amendment. 


464 


95.7 


0.42 




57 


How is each state’s number of Representatives 
decided? 


811 


78.3 


0.66 


o 


58 


Name one responsibility that is only for United 
States citizens. 


462 


96.3 


0.29 




59 


Who is the Speaker of the House of 
Representatives now? 


462 


76.4 


0.58 


o 


60 


What does the judicial branch do? 


461 


84.2 


0.42 


3S 


61 


Name one U.S. territory. 


809 


81.0 


0.63 


3S 


62 


Why does the flag have 13 stripes? 


462 


98.7 


0.16 


* 


63 


What does freedom of religion mean? 


808 


89.9 


0.54 


• 


64 


Name one thing only the federal government can 
do. 


462 


77.7 


0.57 


o 


65 


What did the Emancipation Proclamation do? 


462 


84.0 


0.29 


3S 


66 


When must all males register for the Selective 
Service? 


462 


95.0 


0.44 
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Item 

number 


Item text 


N 


Percent 

correct 


Item 

discrimination 


Flag 


67 


What are the two major political parties in the U.S. 
today? 


517 


97.3 


0.44 




68 


What country is on the northern border of the 
United States? 


461 


92.4 


0.55 


♦ 


69 


When was the Constitution drafted? 


460 


78.0 


0.37 


O 


70 


Who is the Senate Majority Leader now? 


460 


59.1 


0.55 


□ 


71 


Where did most of America’s colonists come from 
before the Revolution? 


870 


84.7 


0.64 


3S 


72 


What did the Declaration of Independence do? 


464 


79.5 


0.56 


O 


73 


Why did the colonists fight the British? 


465 


75.9 


0.62 


o 


74 


Name one example of checks and balances. 


463 


71.9 


0.67 


o 


75 


Who can vote in the U.S. presidential elections? 


465 


93.8 


0.39 


♦ 


76 


When is the last day you can send in federal 
income tax forms? 


465 


87.7 


0.54 


• 


77 


When was the Declaration of Independence 
adopted? 


465 


89.7 


0.35 


• 


78 


What group of people was taken to America and 
sold as slaves? 


465 


88.0 


0.65 


• 


79 


What country is on the southern border of the 
United States? 


464 


93.1 


0.52 


♦ 


80 


Where is the Grand Canyon? 


462 


82.5 


0.61 




81 


What ocean is on the west coast of the United 
States? 


437 


94.7 


0.49 


♦ 


82 


Name one large mountain range in the United 
States. 


437 


86.0 


0.52 


• 


83 


What does the President’s Cabinet do? 


436 


91.5 


0.49 


♦ 


84 


Who does a U.S. Representative represent? 


437 


79.9 


0.63 


O 


85 


What decides each state’s number of U.S. 
Representatives? 


437 


84.7 


0.68 


3S 


86 


What group of essays supported passage of the 
U.S. Constitution? 


436 


57.6 


0.57 


□ 


87 


Name two national U.S. holidays. 


437 


99.1 


0.31 




88 


What is the capital of the U.S.? 


492 


98.4 


0.25 




89 


Why were the colonists upset with the British 
government? 


433 


84.1 


0.69 


3S 


90 


What was the main concern of the United States 
during the Cold War? 


433 


79.2 


0.67 


O 


91 


What is the “rule of law”? 


428 


70.3 


0.49 


o 


92 


What is the tallest mountain in the United States? 


428 


68.7 


0.59 


pj 


93 


Name your state’s two U.S. Senators. 


485 


83.5 


0.39 


3S 


94 


The United States fought Japan, Germany, and 
Italy during which war? 


430 


94.4 


0.35 


♦ 


95 


Name two ways that Americans can participate in 
their democracy. 


430 


83.7 


0.61 


3S 


96 


Who does a U.S. Senator represent? 


428 


88.8 


0.46 


• 


97 


Name one branch or part of the government. 


430 


97.9 


0.33 




98 


Martin Luther King, Jr. had a dream for America. 
What was his dream? 


429 


94.6 


0.38 


♦ 


99 


What is the political party of the President now? 


427 


95.3 


0.42 




100 


Who is the Commander-in-Chief of the military? 


428 


96.0 


0.33 
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Item 

number 


Item text 


N 


Percent 

correct 


Item 

discrimination 


Flag 


101 


There are four amendments to the Constitution 
about who can vote. Describe one of them. 


426 


91.1 


0.50 


♦ 


102 


Who is the Attorney General now? 


426 


70.4 


0.57 


O 


103 


What is a veto? 


426 


87.1 


0.63 


• 


104 


What do we call the first ten amendments to the 
Constitution? 


427 


93.9 


0.38 


♦ 


105 


We elect a U.S. Senator for how many years? 


484 


96.1 


0.27 




106 


Name one war fought by the United States in the 
1800s. 


483 


87.2 


0.57 


• 


107 


What does the Constitution do? 


424 


79.7 


0.41 


o 


108 


Name one thing Benjamin Franklin is famous for. 


422 


71.3 


0.66 


o 


109 


What is the supreme law of the land? 


481 


94.6 


0.36 


♦ 


110 


What did Susan B. Anthony do? 


767 


74.6 


0.67 


o 


111 


Name one of the things that Abraham Lincoln did. 


411 


90.8 


0.41 


♦ 


112 


If the President can no longer serve, who becomes 
President? 


412 


99.0 


0.28 




113 


Why does the flag have 50 stars? 


412 


99.8 


0.29 




114 


What Cabinet-level agency advises the President 
on foreign policy? 


411 


69.1 


0.54 


□ 


115 


What is the name of the Vice President of the 
United States? 


468 


98.9 


0.25 




116 


What is one thing a state government can do? 


411 


80.8 


0.52 


3S 


117 


Who vetoes bills? 


468 


94.2 


0.40 


♦ 


118 


How many U.S. Senators does each state have? 


412 


97.8 


0.38 




119 


How old must a President be? 


412 


96.1 


0.36 




120 


There are 13 original states. Name three. 


410 


96.8 


0.22 


* 


121 


What alliance of North America and European 
countries was created during the Cold War? 


713 


68.7 


0.68 


□ 


122 


Name one of the two longest rivers in the United 
States. 


370 


93.5 


0.57 


♦ 


123 


Who is called the “Father of Our Country”? 


370 


98.4 


0.16 


* 


124 


Why do we have three branches of government? 


423 


86.5 


0.65 


• 


125 


Who was President during World War 1? 


367 


68.1 


0.51 


□ 


126 


What international organization was established 
after World War II (WWII) to keep the world at 
peace? 


369 


86.7 


0.59 


• 


127 


Name one promise you make when you say the 
Oath of Allegiance. 


368 


90.5 


0.57 


♦ 


128 


Who nominates justices to the Supreme Court? 


370 


95.1 


0.26 




129 


Who is in charge of the executive branch? 


369 


94.9 


0.37 


♦ 


130 


Name one state that borders Canada. 


369 


94.6 


0.43 


♦ 


131 


What does “We the People” mean in the 
Constitution? 


355 


74.6 


0.59 


o 


132 


What territory did the United States buy from 
France in 1803? 


355 


83.4 


0.61 


3S 


133 


What is the name of the President of the United 
States? 


355 


100 


1.00 




134 


What did the abolitionists try to end before the Civil 
War? 


355 


89.0 


0.52 


• 
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Percent 


Item 
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correct 
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Flag 


135 


We elect a U.S. Representative for how many 
years? 


355 


92.7 


0.31 


♦ 


136 


Who governs the people in a self-governed 
country? 


700 


79.6 


0.69 


O 


137 


Who confirms Supreme Court justices? 


354 


70.6 


0.59 


o 


138 


What is the highest court in the U.S.? 


353 


98.9 


0.11 


* 


139 


Where is the Statue of Liberty? 


407 


94.3 


0.31 


♦ 


140 


Who is the Governor of your state? 


351 


93.7 


0.35 


♦ 


141 


What type of economic system does the U.S. have? 


346 


81.5 


0.71 




142 


On the Fourth of July we celebrate independence 
from what country? 


348 


98.6 


0.23 


* 



Key 


90.0-94.9% 


♦ 


85.0-89.9% 


• 


80.0-84.9% 


X 


70.0-79.9% 


O 


60.0-69.9% 


□ 


< 60.0% 


▲ 


Item Discrimination below 0.25 


* 
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Table 4.3: Summary Statistics for New Civics Test Items for Current Applicants 



Civics items 




Percent Correct 


N 


95.0-100% 


42 


90.0-94.9% 


28 


85.0-89.9% 


21 


80.0-84.9% 


16 


70.0-79.9% 


23 


60.0-69.9% 


10 


< 60.0% 


2 


Total 


142 


Item Discrimination below 0.25 


12 
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Item 

number 


Item text 


N 


Percent 

correct 


Item 

discrimination 


Flag 


1 


[REDACTED] 


595 


99.5 


0.43 




2 


[REDACTED] 


583 


99.0 


0.71 




3 


[REDACTED] 


583 


99.3 


0.76 




4 


[REDACTED] 


590 


99.3 


0.47 




5 


[REDACTED] 


577 


99.3 


0.85 




6 


[REDACTED] 


572 


99.3 


0.97 




7 


[REDACTED] 


564 


100 


1.00 




8 


[REDACTED] 1 


1011 


99.7 


0.48 




9 


[REDACTED] 2 


1047 


99.8 


0.25 




10 


[REDACTED] 


549 


99.6 


0.30 




11 


[REDACTED] 


537 


99.6 


0.59 




12 


[REDACTED] 


538 


99.3 


0.63 




13 


[REDACTED] 


549 


98.9 


-0.02 


* 


14 


[REDACTED] 


542 


99.8 


0.26 




15 


[REDACTED] 


538 


99.6 


0.23 


* 


16 


[REDACTED] 


536 


99.8 


^goil 


* 


17 


[REDACTED] 


525 


99.8 


0.00 


* 


18 


[REDACTED] 


525 


99.8 


0.00 


* 


19 


[REDACTED] 


531 


99.8 


-0.01 


* 


20 


[REDACTED] 


522 


99.6 


-0.01 


* 


21 


[REDACTED] 


520 


99.6 


-0.01 


* 


22 


[REDACTED] 


542 


99.3 


0.23 


* 


23 


[REDACTED] 


523 


99.0 


0.44 




24 


[REDACTED] 


530 


99.2 


0.40 




25 


[REDACTED] 


509 


100 


1.00 




26 


[REDACTED] 


497 


99.8 


0.33 




27 


[REDACTED] 


495 


99.4 


0.25 




28 


[REDACTED] 


506 


99.8 


0.24 


* 


29 


[REDACTED] 


483 


97.9 


0.25 




30 


[REDACTED] 


490 


99.2 


0.22 


* 


31 


[REDACTED] 


489 


100 


1.00 




32 


[REDACTED] 


488 


99.2 


0.38 




33 


[REDACTED] 


471 


99.8 


0.29 




34 


[REDACTED] 


463 


99.4 


0.48 





1 This item appeared on more than one form. N, percent correct, and item discrimination for this item are based on combined data. 

2 This item appeared on more than one form. N, percent correct, and item discrimination for this item are based on combined data. 



188 





Naturalization Test Redesign Project 
Civics Item Selection Analysis 



List of Supporting Tables 



Key 


90.0-94.9% 


♦ 


85.0-89.9% 


• 


80.0-84.9% 


X 


70.0-79.9% 


O 


60.0-69.9% 


□ 


< 60.0% 


▲ 


Item Discrimination below 0.25 


* 
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Table 4.5: Summary Statistics for New Reading Test Items for Current 
Applicants 



Reading Items 




Percent Correct 


N 


95.0-100% 


34 


90.0-94.9% 


0 


85.0-89.9% 


0 


80.0-84.9% 


0 


70.0-79.9% 


0 


60.0-69.9% 


0 


< 60.0% 


0 


Total 


34 


Item Discrimination below 0.25 


11 
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Item 

number 


Item text 


N 


Percent 

correct 


Item 

discrimination 


Flag 


1 


[REDACTED] 


600 


95.0 


0.34 




2 


[REDACTED] 


581 


95.9 


0.37 




3 


[REDACTED] 


576 


96.9 


0.43 




4 


[REDACTED] 


591 


96.3 


0.42 




5 


[REDACTED] 


570 


97.2 


0.65 




6 


[REDACTED] 


561 


95.9 


0.67 




7 


[REDACTED] 


563 


97.7 


0.35 




8 


[REDACTED] 1 


1,011 


98.0 


0.50 




9 


[REDACTED] 2 


1,039 


96.5 


0.39 




10 


[REDACTED] 


551 


96.2 


0.36 




11 


[REDACTED] 


536 


97.8 


0.57 




12 


[REDACTED] 


533 


97.6 


0.68 




13 


[REDACTED] 


551 


96.4 


0.39 




14 


[REDACTED] 


540 


97.2 


0.55 




15 


[REDACTED] 


531 


98.3 


0.41 




16 


[REDACTED] 


539 


95.4 


0.35 




17 


[REDACTED] 


526 


97.5 


0.47 




18 


[REDACTED] 


521 


97.1 


0.52 




19 


[REDACTED] 


535 


95.0 


0.13 


* 


20 


[REDACTED] 


518 


97.9 


0.24 


* 


21 


[REDACTED] 


511 


97.8 


0.22 


* 


22 


[REDACTED] 


546 


97.6 


0.44 




23 


[REDACTED] 


522 


95.6 


0.61 




24 


[REDACTED] 


522 


96.7 


0.60 




25 


[REDACTED] 


513 


96.9 


0.19 


* 


26 


[REDACTED] 


498 


98.8 


0.37 




27 


[REDACTED] 


492 


96.5 


0.41 




28 


[REDACTED] 


508 


96.5 


0.29 




29 


[REDACTED] 


480 


97.1 


0.44 




30 


[REDACTED] 


496 


97.2 


0.18 


•k 


31 


[REDACTED] 


488 


97.3 


0.13 


* 


32 


[REDACTED] 


483 


97.7 


0.34 




33 


[REDACTED] 


482 


94.0 


0.48 


♦ 


34 


[REDACTED] 


470 


94.5 


0.61 


♦ 


1 This item appeared on more than one form. N, percent correct, and item discrimination for this item are based on combined data. 


2 This item appeared on more than one form. N , percent correct, and item discrimination for this item are based on combined data. 
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90.0-94.9% 


♦ 


85.0-89.9% 


• 


80.0-84.9% 


X 


70.0-79.9% 


o 


60.0-69.9% 


□ 


< 60.0% 


▲ 


Item Discrimination below 0.25 


* 
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Writing items 




Percent correct 


N 


95.0-100% 


32 


90.0-94.9% 


2 


85.0-89.9% 


0 


80.0-84.9% 


0 


70.0-79.9% 


0 


60.0-69.9% 


0 


< 60.0% 


0 


Total 


34 


Item Discrimination below 0.25 


6 
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Item 

number 


Item text 


N 


Percent 

correct 


Flag 


1 


Who lived in America before the Europeans arrived? 


155 


63.9 


• 


2 


Name one of the writers of the Federalist Papers. 


141 


8.5 


— □ 


3 


Name two rights that are only for United States citizens. 


137 


34.3 


O 


4 


Who is the President now? 


155 


90.3 




5 


Who makes federal laws? 


138 


61.6 


• 


6 


Who signs bills to become laws? 


139 


74.1 


♦ 


7 


What are the two parts of the United States Congress? 


130 


33.1 


o 


8 


What is the political party of the majority in the Senate now? 


130 


53.1 


• 


9 


What is the majority political party in the House of Representatives 
now? 


152 


43.4 


X 


10 


What do we call changes to the Constitution? 


152 


57.2 


• 


11 


What are the three branches or parts of the government? 


124 


63.7 


• 


12 


Name one important idea found in the Declaration of Independence. 


125 


21.6 


— o 


13 


Name two Cabinet-level positions. 


140 


20.0 


o 


14 


To become President of the United States, what must the person be 
at birth? 


118 


46.6 


X 


15 


Name one war fought by the United States in the 1 900s. 


140 


43.6 


X 


16 


What did Martin Luther King, Jr. do? 


144 


56.3 


• 


17 


Name two rights of everyone living in the U.S. 


119 


31.9 


o 


18 


The House of Representatives has how many voting members? 


130 


51.5 


• 


19 


Who is the Vice President now? 


129 


69.0 


♦ 


20 


Is the current President in his first or second term? 


124 


79.8 


♦ 


21 


Name one problem that led to the Civil War. 


131 


22.1 


o 


22 


What are “inalienable rights”? 


136 


5.9 


□ 


23 


Who becomes President if both the President and the Vice President 
can no longer serve? 


144 


52.1 


• 


24 


What is the capital (or capital city) of your state? 


161 


81.4 




25 


How many United States Senators are there? 


155 


67.1 


♦ 


26 


What is the name of the National Anthem? 


161 


42.9 


X 


27 


What does it mean that the U.S. Constitution is a constitution of 
limited powers? 


135 


3.7 


□ 


28 


Who was President during the Great Depression and World War II? 


136 


21.3 


o 


29 


Who is the Secretary of State now? 


159 


31.4 


o 


30 


What is an amendment? 


138 


34.1 


o 


31 


What country sold the Louisiana Territory to the United States? 


117 


25.6 


o 


32 


Name your U.S. Representative. 


118 


23.7 


o 


33 


Name the U.S. war between the North and the South. 


159 


35.2 


X 


34 


Who wrote the Declaration of Independence? 


116 


52.6 


• 


35 


We elect a President for how many years? 


139 


91.4 




36 


In 1803, the United States bought a large amount of land from 
France. Where was that land? 


137 


21.2 


o 


37 


Who is the Chief Justice of the United States? 


124 


39.5 


X 


38 


Name two of the natural, or inalienable, rights in the Declaration of 
Independence. 


115 


12.2 


□ 


39 


Name the state that is in the middle of the Pacific Ocean. 


125 


40.0 


X 


40 


What major event happened on September 1 1 , 2001 , in the United 


116 


74.1 


♦ 
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Flag 


41 


Name one reason why the colonists came to America? 


154 


68.2 


♦ 


42 


What is the Pledge of Allegiance? 


120 


10.8 


pj 


43 


How many justices are on the Supreme Court? 


118 


60.2 


• 


44 


Why do we have 1 3 stripes on the flag? 


138 


58.0 


• 


45 


How many amendments does the Constitution have? 


130 


66.9 


♦ 


46 


Which U.S. World War II general later became President? 


131 


15.3 


□ 


47 


Who was the first President? 


119 


89.1 




48 


What happened at the Constitutional Convention? 


113 


8.0 


□ 


49 


Name one state that borders on Mexico. 


125 


61.6 


• 


50 


How many full terms can a President serve? 


145 


81.4 




51 


What movement tried to end racial discrimination? 


120 


16.7 


□ 


52 


Name one of the major American Indian tribes in the United States. 


154 


42.9 




53 


What is self-government? 


150 


22.7 


o 


54 


When do we celebrate Independence Day? 


133 


89.5 




55 


In what month are general presidential elections held in the United 
States? 


137 


73.0 


♦ 


56 


Name one right or freedom from the First Amendment. 


139 


51.1 


• 


57 


How is each state’s number of Representatives decided? 


154 


24.0 


o 


58 


Name one responsibility that is only for United States citizens. 


121 


57.0 


• 


59 


Who is the Speaker of the House of Representatives now? 


122 


32.8 


o 


60 


What does the judicial branch do? 


121 


38.8 




61 


Name one U.S. territory. 


179 


34.1 


O 


62 


Why does the flag have 13 stripes? 


147 


72.1 


♦ 


63 


What does freedom of religion mean? 


168 


30.4 


o 


64 


Name one thing only the federal government can do. 


134 


19.4 


□ 


65 


What did the Emancipation Proclamation do? 


140 


41.4 




66 


When must all males register for the Selective Service? 


136 


50.0 


• 


67 


What are the two major political parties in the U.S. today? 


158 


67.1 


♦ 


68 


What country is on the northern border of the United States? 


164 


65.2 


♦ 


69 


When was the Constitution drafted? 


145 


26.9 


O 


70 


Who is the Senate Majority Leader now? 


151 


11.3 


□ 


71 


Where did most of America’s colonists come from before the 
Revolution? 


169 


45.6 


3S 


72 


What did the Declaration of Independence do? 


139 


30.9 


O 


73 


Why did the colonists fight the British? 


149 


31.5 


o 


74 


Name one example of checks and balances. 


144 


16.0 


□ 


75 


Who can vote in the U.S. presidential elections? 


143 


21.0 


o 


76 


When is the last day you can send in federal income tax forms? 


142 


51.4 


• 


77 


When was the Declaration of Independence adopted? 


137 


65.0 


♦ 


78 


What group of people was taken to America and sold as slaves? 


135 


39.3 




79 


What country is on the southern border of the United States? 


132 


64.4 


• 


80 


Where is the Grand Canyon? 


132 


37.1 


3€ 


81 


What ocean is on the west coast of the United States? 


169 


55.6 


• 


82 


Name one large mountain range in the United States. 


161 


26.1 


O 


83 


What does the President’s Cabinet do? 


165 


32.7 


o 


84 


Who does a U.S. Representative represent? 


182 


22.5 


o 


85 


What decides each state’s number of U.S. Representatives? 


165 


24.8 


o 


86 


What group of essays supported passage of the U.S. Constitution? 


157 


9.6 


□ 


87 


Name two national U.S. holidays. 


166 


78.3 


♦ 
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Item text 


N 


Percent 

correct 


Flag 


88 


What is the capital of the U.S.? 


175 


85.7 




89 


Why were the colonists upset with the British government? 


165 


27.3 


O 


90 


What was the main concern of the United States during the Cold 
War? 


164 


14.6 


□ 


91 


What is the “rule of law”? 


165 


21.2 


o 


92 


What is the tallest mountain in the United States? 


167 


18.0 


pi 


93 


Name your state’s two U.S. Senators. 


175 


42.9 


X 


94 


The United States fought Japan, Germany, and Italy during which 
war? 


164 


62.8 


• 


95 


Name two ways that Americans can participate in their democracy. 


182 


18.7 


□ 


96 


Who does a U.S. Senator represent? 


160 


20.0 


o 


97 


Name one branch or part of the government. 


168 


71.4 


♦ 


98 


Martin Luther King, Jr. had a dream for America. What was his 
dream? 


183 


48.1 


X 


99 


What is the political party of the President now? 


167 


64.7 


• 


100 


Who is the Commander-in-Chief of the military? 


182 


71.4 


♦ 


101 


There are four amendments to the Constitution about who can vote. 
Describe one of them. 


227 


21.6 


o 


102 


Who is the Attorney General now? 


234 


22.6 


o 


103 


What is a veto? 


229 


37.6 


X 


104 


What do we call the first ten amendments to the Constitution? 


228 


54.8 


• 


105 


We elect a U.S. Senator for how many years? 


207 


67.6 


♦ 


106 


Name one war fought by the United States in the 1 800s. 


213 


38.5 


X 


107 


What does the Constitution do? 


228 


34.6 


o 


108 


Name one thing Benjamin Franklin is famous for. 


227 


13.7 


pi 


109 


What is the supreme law of the land? 


207 


52.7 


• 


110 


What did Susan B. Anthony do? 


262 


30.2 


o 


111 


Name one of the things that Abraham Lincoln did. 


184 


57.1 


• 


112 


If the President can no longer serve, who becomes President? 


189 


66.7 


♦ 


113 


Why does the flag have 50 stars? 


200 


85.5 




114 


What Cabinet-level agency advises the President on foreign policy? 


197 


13.7 


pj 


115 


What is the name of the Vice President of the United States? 


192 


71.4 


♦ 


116 


What is one thing a state government can do? 


184 


21.2 


o 


117 


Who vetoes bills? 


178 


55.6 


• 


118 


How many U.S. Senators does each state have? 


200 


76.5 


♦ 


119 


How old must a President be? 


185 


69.2 


♦ 


120 


There are 13 original states. Name three. 


205 


74.6 


♦ 


121 


What alliance of North America and European countries was created 
during the Cold War? 


161 


18.0 


□ 


122 


Name one of the two longest rivers in the United States. 


175 


60.0 


• 


123 


Who is called the “Father of Our Country”? 


160 


74.4 


♦ 


124 


Why do we have three branches of government? 


153 


30.7 


o 


125 


Who was President during World War 1? 


159 


10.7 


□ 


126 


What international organization was established after World War II 
(WWII) to keep the world at peace? 


160 


31.9 


o 


127 


Name one promise you make when you say the Oath of Allegiance. 


152 


26.3 


o 


128 


Who nominates justices to the Supreme Court? 


176 


61.4 


• 


129 


Who is in charge of the executive branch? 


175 


52.6 


• 


130 


Name one state that borders Canada. 


176 


52.8 


• 
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Item 

number Item text 


N 


Percent 

correct 


Flag 


131 


What does “We the People” mean in the Constitution? 


151 


13.9 


□ 


132 


What territory did the United States buy from France in 1803? 


144 


34.0 


O 


133 


What is the name of the President of the United States? 


158 


92.4 


134 


What did the abolitionists try to end before the Civil War? 


151 


25.2 


O 


135 


We elect a U.S. Representative for how many years? 


151 


59.6 


• 


136 


Who governs the people in a self-governed country? 


184 


16.8 


□! 


137 


Who confirms Supreme Court justices? 


145 


11.0 


□ 


138 


What is the highest court in the U.S.? 


151 


70.2 


♦ 


139 


Where is the Statue of Liberty? 


152 


67.1 


♦ 


140 


Who is the Governor of your state? 


145 


60.7 


• 


141 


What type of economic system does the U.S. have? 


142 


23.9 


o 


142 


On the Fourth of July we celebrate independence from what country? 


142 


67.6 


♦ 



Key 


65.0-79.9% 


♦ 


50.0-64.9% 


• 


35.0-49.9% 


X 


20.0-34.9% 


O 


10.0-19.9% 


□ 


< 1 0.0% 


▲ 
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Table 4.9: Summary Statistics for New Civics Test Items for Prospective 
Applicants 



Civics items 




Percent Correct 


N 


80 . 0 - 100 . 0 % 


9 


65 . 0 - 79 . 9 % 


25 


50 . 0 - 64 . 9 % 


30 


35 . 0 - 49 . 9 % 


17 | 


20 . 0 - 34 . 9 % 


39 


10 . 0 - 19 . 9 % 


17 


< 10 . 0 % 


5 


Total 


142 
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Table 4.10: Item Properties of New Reading Test Items for Prospective Applicants 



Item 

number 


Item text 


N 


Percent 

correct 


Flag 


1 


[REDACTED] 


107 


96.3 




2 


[REDACTED] 


106 


98.1 




3 


[REDACTED] 


106 


99.1 




4 


[REDACTED] 


108 


88.9 




5 


[REDACTED] 


105 


99.0 




6 


[REDACTED] 


105 


94.3 




7 


[REDACTED] 


105 


100.0 




8 


[REDACTED] 


207 


95.2 




9 


[REDACTED] 


207 


96.6 




10 


[REDACTED] 


107 


95.3 




11 


[REDACTED] 


105 


98.1 




12 


[REDACTED] 


105 


95.2 




13 


[REDACTED] 


105 


85.7 




14 


[REDACTED] 


109 


97.2 




15 


[REDACTED] 


106 


96.2 




16 


[REDACTED] 


107 


98.1 




17 


[REDACTED] 


109 


96.3 




18 


[REDACTED] 


106 


96.2 




19 


[REDACTED] 


111 


93.7 




20 


[REDACTED] 


106 


96.2 




21 


[REDACTED] 


106 


99.1 




22 


[REDACTED] 


106 


95.3 




23 


[REDACTED] 


106 


97.2 




24 


[REDACTED] 


107 


96.3 




25 


[REDACTED] 


105 


99.0 




26 


[REDACTED] 


101 


99.0 




27 


[REDACTED] 


106 


83.0 




28 


[REDACTED] 


101 


96.0 




29 


[REDACTED] 


101 


74.3 


♦ 


30 


[REDACTED] 


105 


93.3 




31 


[REDACTED] 


101 


97.0 




32 


[REDACTED] 


100 


86.0 




33 


[REDACTED] 


105 


96.2 




34 


[REDACTED] 


100 


92.0 
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Key 


65 . 0 - 79 . 9 % 


♦ 


50 . 0 - 64 . 9 % 


• 


35 . 0 - 49 . 9 % 


X 


20 . 0 - 34 . 9 % 


o 


10 . 0 - 19 . 9 % 


□ 


< 1 0 . 0 % 


▲ 
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Table 4.11 : Summary Statistics for New Reading Test Items for Prospective 
Applicants 



Reading Items 


Percent Correct 


N 


80 . 0 - 100 . 0 % 


33 


65 . 0 - 79 . 9 % 


1 


50 . 0 - 64 . 9 % 


0 


35 . 0 - 49 . 9 % 


0 


20 . 0 - 34 . 9 % 


0 


10 . 0 - 19 . 9 % 


0 


< 10 . 0 % 


0 


Total 


34 
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Item 

number 


Item text 


N 


Percent 

correct 


Flag 


1 


[REDACTED] 


276 


55.8 


• 


2 


[REDACTED] 


131 


65.6 


♦ 


3 


[REDACTED] 


298 


81.2 




4 


[REDACTED] 


283 


75.3 


♦ 


5 


[REDACTED] 


260 


81.5 




6 


[REDACTED] 


139 


71.9 


♦ 


7 


[REDACTED] 


335 


72.5 


♦ 


8 


[REDACTED] 


396 


64.9 


• 


9 


[REDACTED] 


132 


65.9 


♦ 


10 


[REDACTED] 


229 


68.6 


♦ 


11 


[REDACTED] 


203 


68.0 


♦ 


12 


[REDACTED] 


111 


55.0 


• 


13 


[REDACTED] 


137 


52.6 


• 


14 


[REDACTED] 


274 


73.0 


♦ 


15 


[REDACTED] 


242 


64.0 


• 


16 


[REDACTED] 


245 


62.0 


• 


17 


[REDACTED] 


288 


68.4 


♦ 


18 


[REDACTED] 


269 


68.0 


♦ 


19 


[REDACTED] 


147 


47.6 


& 


20 


[REDACTED] 


282 


81.2 




21 


[REDACTED] 


152 


60.5 


• 


22 


[REDACTED] 


294 


78.6 


♦ 


23 


[REDACTED] 


144 


56.9 


• 


24 


[REDACTED] 


275 


77.1 


♦ 


25 


[REDACTED] 


275 


71.3 


♦ 


26 


[REDACTED] 


291 


74.9 


♦ 


27 


[REDACTED] 


227 


34.4 


O 


28 


[REDACTED] 


176 


45.5 


K 


29 


[REDACTED] 


285 


54.0 


• 


30 


[REDACTED] 


204 


58.3 


• 


31 


[REDACTED] 


271 


70.8 


♦ 


32 


[REDACTED] 


274 


66.1 


♦ 


33 


[REDACTED] 


208 


62.0 


• 


34 


[REDACTED] 


160 


45.0 
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Key 


65 . 0 - 79 . 9 % 


♦ 


50 . 0 - 64 . 9 % 


• 


35 . 0 - 49 . 9 % 


X 


20 . 0 - 34 . 9 % 


o 


10 . 0 - 19 . 9 % 


□ 


< 1 0 . 0 % 


▲ 
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Table 4.13: Summary Statistics for New Writing Test Items for Prospective 
Applicants 



Writing items 


Percent correct 


N 


80 . 0 - 100 . 0 % 


3 


65 . 0 - 79 . 9 % 


16 


50 . 0 - 64 . 9 % 


11 


35 . 0 - 49 . 9 % 


3 


20 . 0 - 34 . 9 % 


1 


10 . 0 - 19 . 9 % 


0 


< 10 . 0 % 


0 


Total 


34 
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Table 4.14: Correlations Between Current and Prospective Applicants for Civics, 
Reading, and Writing Items 



Item type 


Correlation 


Civics 


.86 


Reading 


.69 


Writing 


.43 
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Table 4.15: Average Percentage of Items Answered Correctly, by Sample 



Sample 


Civics 


Reading 


Writing 


Current applicants 


87.4 


99.5 


96.8 


Prospective applicants 


44.5 


94.7 


64.7 


Combined sample of current and prospective 
applicants 


65.9 


97.1 


80.7 
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Table 4.16: Item Properties of New Civics Test Items in Combined Sample of 
Current and Prospective Applicants 



Item 

number 


Item text 


N 

combined 


Avg. 
percent 
correct Flag 


1 


Who lived in America before the Europeans arrived? 


679 


80.3 


♦ 


2 


Name one of the writers of the Federalist Papers. 


665 


38.3 


□ 


3 


Name two rights that are only for United States citizens. 


660 


64.1 




4 


Who is the President now? 


735 


94.9 




5 


Who makes federal laws? 


662 


77.9 


• 


6 


Who signs bills to become laws? 


661 


86.5 


♦ 


7 


What are the two parts of the United States Congress? 


651 


60.4 


3€ 


8 


What is the political party of the majority in the Senate now? 


649 


71.0 


• 


9 


What is the majority political party in the House of 
Representatives now? 


670 


65.5 




10 


What do we call changes to the Constitution? 


670 


76.5 


• 


11 


What are the three branches or parts of the government? 


619 


80.5 


♦ 


12 


Name one important idea found in the Declaration of 
Independence. 


620 


51.9 


O 


13 


Name two Cabinet-level positions. 


635 


50.3 


O 


14 


To become President of the United States, what must the 
person be at birth? 


613 


71.7 


• 


15 


Name one war fought by the United States in the 1 900s. 


635 


67.8 




16 


What did Martin Luther King, Jr. do? 


638 


76.2 


• 


17 


Name two rights of everyone living in the U.S. 


611 


60.8 




18 


The House of Representatives has how many voting members? 


677 


72.3 


• 


19 


Who is the Vice President now? 


675 


83.4 


♦ 


20 


Is the current President in his first or second term? 


612 


88.9 


♦ 


21 


Name one problem that led to the Civil War. 


622 


53.2 


o 


22 


What are “inalienable rights”? 


627 


37.8 


□ 


23 


Who becomes President if both the President and the Vice 
President can no longer serve? 


693 


72.8 


• 


24 


What is the capital (or capital city) of your state? 


653 


90.1 




25 


How many United States Senators are there? 


646 


81.9 


♦ 


26 


What is the name of the National Anthem? 


652 


67.2 




27 


What does it mean that the U.S. Constitution is a constitution of 
limited powers? 


625 


34.4 


□ 


28 


Who was President during the Great Depression and World 
War II? 


625 


48.0 


□ 


29 


Who is the Secretary of State now? 


648 


56.7 


o 


30 


What is an amendment? 


628 


61.1 




31 


What country sold the Louisiana Territory to the United States? 


602 


56.1 


o 


32 


Name your U.S. Representative. 


602 


44.6 


□ 


33 


Name the U.S. war between the North and the South. 


1049 


60.5 


& 


34 


Who wrote the Declaration of Independence? 


601 


71.2 


• 


35 


We elect a President for how many years? 


624 


95.1 




36 


In 1803, the United States bought a large amount of land from 
France. Where was that land? 


621 


54.4 


o 
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Avg. 



Item N percent 



number 


Item text 


combined 


correct 


Flag 


37 


Who is the Chief Justice of the United States? 


664 


59.1 


O 


38 


Name two of the natural, or inalienable, rights in the Declaration 
of Independence. 


597 


37.0 


□ 


39 


Name the state that is in the middle of the Pacific Ocean. 


607 


65.8 


X 


40 


What major event happened on September 1 1 , 2001 , in the 
United States? 


599 


85.4 


♦ 


41 


Name one reason why the colonists came to America? 


634 


81.7 


♦ 


42 


What is the Pledge of Allegiance? 


600 


43.0 


□ 


43 


How many justices are on the Supreme Court? 


654 


75.9 


• 


44 


Why do we have 1 3 stripes on the flag? 


674 


78.6 


• 


45 


How many amendments does the Constitution have? 


610 


80.6 


♦ 


46 


Which U.S. World War II general later became President? 


611 


40.3 


□ 


47 


Who was the first President? 


655 


94.2 




48 


What happened at the Constitutional Convention? 


590 


42.0 


□ 


49 


Name one state that borders on Mexico. 


605 


78.4 


• 


50 


How many full terms can a President serve? 


625 


89.4 


♦ 


51 


What movement tried to end racial discrimination? 


584 


46.6 


_□ 


52 


Name one of the major American Indian tribes in the United 
States. 


618 


64.4 


X 


53 


What is self-government? 


614 


49.8 


□ 


54 


When do we celebrate Independence Day? 


597 


94.7 




55 


In what month are general presidential elections held in the 
United States? 


601 


85.7 


♦ 


56 


Name one right or freedom from the First Amendment. 


603 


73.4 


• 


57 


How is each state’s number of Representatives decided? 


965 


51.2 


o 


58 


Name one responsibility that is only for United States citizens. 


583 


76.7 


• 


59 


Who is the Speaker of the House of Representatives now? 


584 


54.6 


o 


60 


What does the judicial branch do? 


582 


61.5 


X 


61 


Name one U.S. territory. 


988 


57.6 


o 


62 


Why does the flag have 1 3 stripes? 


609 


85.4 


♦ 


63 


What does freedom of religion mean? 


976 


60.2 


X 


64 


Name one thing only the federal government can do. 


596 


48.6 


□ 


65 


What did the Emancipation Proclamation do? 


602 


62.7 


X 


66 


When must all males register for the Selective Service? 


598 


72.5 


• 


67 


What are the two major political parties in the U.S. today? 


675 


82.2 


♦ 


68 


What country is on the northern border of the United States? 


625 


78.8 


• 


69 


When was the Constitution drafted? 


605 


52.5 


o 


70 


Who is the Senate Majority Leader now? 


611 


35.2 


□ 


71 


Where did most of America’s colonists come from before the 
Revolution? 


1039 


65.2 


X 


72 


What did the Declaration of Independence do? 


603 


55.2 


o 


73 


Why did the colonists fight the British? 


614 


53.7 


o 


74 


Name one example of checks and balances. 


607 


44.0 


□ 


75 


Who can vote in the U.S. presidential elections? 


608 


57.4 


o 


76 


When is the last day you can send in federal income tax forms? 


607 


69.6 


X 


77 


When was the Declaration of Independence adopted? 


602 


77.4 


• 


78 


What group of people was taken to America and sold as 
slaves? 


600 


63.7 


X 
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correct 


Flag 


79 


What country is on the southern border of the United States? 


596 


78.8 


• 


80 


Where is the Grand Canyon? 


594 


59.8 


O 


81 


What ocean is on the west coast of the United States? 


606 


75.2 


• 


82 


Name one large mountain range in the United States. 


598 


56.1 


o 


83 


What does the President’s Cabinet do? 


601 


62.1 


X 


84 


Who does a U.S. Representative represent? 


619 


51.2 


o 


85 


What decides each state’s number of U.S. Representatives? 


602 


54.8 


o 


86 


What group of essays supported passage of the U.S. 
Constitution? 


593 


33.6 


□ 


87 


Name two national U.S. holidays. 


603 


88.7 


♦ 


88 


What is the capital of the U.S.? 


667 


92.1 




89 


Why were the colonists upset with the British government? 


598 


55.7 


o 


90 


What was the main concern of the United States during the 
Cold War? 


597 


46.9 


□ 


91 


What is the “rule of law”? 


593 


45.8 


□ 


92 


What is the tallest mountain in the United States? 


595 


43.4 


□ 


93 


Name your state’s two U.S. Senators. 


660 


63.2 




94 


The United States fought Japan, Germany, and Italy during 
which war? 


594 


78.6 


• 


95 


Name two ways that Americans can participate in their 
democracy. 


612 


51.2 


o 


96 


Who does a U.S. Senator represent? 


588 


54.4 


o 


97 


Name one branch or part of the government. 


598 


84.7 


♦ 


98 


Martin Luther King, Jr. had a dream for America. What was his 
dream? 


612 


71.4 


• 


99 


What is the political party of the President now? 


594 


80.0 


♦ 


100 


Who is the Commander-in-Chief of the military? 


610 


83.7 


♦ 


101 


There are four amendments to the Constitution about who can 
vote. Describe one of them. 


653 


56.4 


o 


102 


Who is the Attorney General now? 


660 


46.5 


□ 


103 


What is a veto? 


655 


62.4 




104 


What do we call the first ten amendments to the Constitution? 


655 


74.4 


• 


105 


We elect a U.S. Senator for how many years? 


691 


81.9 


♦ 


106 


Name one war fought by the United States in the 1 800s. 


696 


62.9 




107 


What does the Constitution do? 


652 


57.2 


o 


108 


Name one thing Benjamin Franklin is famous for. 


649 


42.5 


□ 


109 


What is the supreme law of the land? 


688 


73.7 


• 


110 


What did Susan B. Anthony do? 


1029 


52.4 


o 


111 


Name one of the things that Abraham Lincoln did. 


595 


74.0 


• 


112 


If the President can no longer serve, who becomes President? 


601 


82.9 


♦ 


113 


Why does the flag have 50 stars? 


612 


92.7 




114 


What Cabinet-level agency advises the President on foreign 
policy? 


608 


41.4 


□ 


115 


What is the name of the Vice President of the United States? 


660 


85.2 


♦ 


116 


What is one thing a state government can do? 


595 


51.0 


Q 


117 


Who vetoes bills? 


646 


74.9 


• 


118 


How many U.S. Senators does each state have? 


612 


87.2 


♦ 
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Item text 
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percent 

correct 


Flag 


119 


How old must a President be? 


597 


82.7 


♦ 


120 


There are 13 original states. Name three. 


615 


85.7 


♦ 


121 


What alliance of North America and European countries was 
created during the Cold War? 


874 


43.4 


□ 


122 


Name one of the two longest rivers in the United States. 


545 


76.8 


• 


123 


Who is called the “Father of Our Country”? 


530 


86.4 


♦ 


124 


Why do we have three branches of government? 


576 


58.6 


O 


125 


Who was President during World War 1? 


526 


39.4 


□ 


126 


What international organization was established after World 
War II (WWII) to keep the world at peace? 


529 


59.3 


o 


127 


Name one promise you make when you say the Oath of 
Allegiance. 


520 


58.4 


o 


128 


Who nominates justices to the Supreme Court? 


546 


78.3 


• 


129 


Who is in charge of the executive branch? 


544 


73.8 


• 


130 


Name one state that borders Canada. 


545 


73.7 


• 


131 


What does “We the People” mean in the Constitution? 


506 


44.3 


□ 


132 


What territory did the United States buy from France in 1803? 


499 


58.7 


o 


133 


What is the name of the President of the United States? 


513 


96.2 




134 


What did the abolitionists try to end before the Civil War? 


506 


57.1 


o 


135 


We elect a U.S. Representative for how many years? 


506 


76.2 


• 


136 


Who governs the people in a self-governed country? 


884 


48.2 


□ 


137 


Who confirms Supreme Court justices? 


499 


40.8 


□ 


138 


What is the highest court in the U.S.? 


504 


84.6 


♦ 


139 


Where is the Statue of Liberty? 


559 


80.7 


♦ 


140 


Who is the Governor of your state? 


496 


77.2 


• 


141 


What type of economic system does the U.S. have? 


488 


52.7 


o 


142 


On the Fourth of July we celebrate independence from what 
country? 


490 


83.1 


♦ 



Key 


80.0-89.9% 


♦ 


70.0-79.9% 


• 


60.0-69.9% 


X 


50.0-59.9% 


O 


40.0-49.9% 


□ 


< 40.0% 


▲ 



210 





Naturalization Test Redesign Project 
Civics Item Selection Analysis 



List of Supporting Tables 



Table 4.17: Summary Statistics for New Civics Test Items in Combined Sample of 
Current and Prospective Applicants 



Civics items 


Percent correct 


N 


90 . 0 - 100 . 0 % 


8 


80 . 0 - 89 . 9 % 


27 


70 . 0 - 79 . 9 % 


30 


60 . 0 - 69 . 9 % 


20 


50 . 0 - 59 . 9 % 


31 


40 . 0 - 49 . 9 % 


19 


< 40 . 0 % 


7 


Total 


142 
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Table 4.18: Item Properties of New Reading Test Items in Combined Sample of 
Current and Prospective Applicants 



Item 

number 


Item text 


N 

combined 


Avg. 

percent 

correct Flag 


1 


[REDACTED] 


702 


97.9 


2 


[REDACTED] 


689 


98.6 


3 


[REDACTED] 


689 


99.2 


4 


[REDACTED] 


698 


94.1 


5 


[REDACTED] 


682 


99.2 


6 


[REDACTED] 


677 


96.8 


7 


[REDACTED] 


669 


100.0 


8 


[REDACTED] 


1218 


97.5 


9 


[REDACTED] 


1254 


98.2 


10 


[REDACTED] 


656 


97.5 


11 


[REDACTED] 


642 


98.9 


12 


[REDACTED] 


643 


97.3 


13 


[REDACTED] 


654 


92.3 


14 


[REDACTED] 


651 


98.5 


15 


[REDACTED] 


644 


97.9 


16 


[REDACTED] 


643 


99.0 


17 


[REDACTED] 


634 


98.1 


18 


[REDACTED] 


631 


98.0 


19 


[REDACTED] 


642 


96.8 


20 


[REDACTED] 


628 


97.9 


21 


[REDACTED] 


626 


99.4 


22 


[REDACTED] 


648 


97.3 


23 


[REDACTED] 


629 


98.1 


24 


[REDACTED] 


637 


97.8 


25 


[REDACTED] 


614 


99.5 


26 


[REDACTED] 


598 


99.4 


27 


[REDACTED] 


601 


91.2 


28 


[REDACTED] 


607 


97.9 


29 


[REDACTED] 


584 


86.1 ♦ 


30 


[REDACTED] 


595 


96.3 


31 


[REDACTED] 


590 


98.5 


32 


[REDACTED] 


588 


92.6 


33 


[REDACTED] 


576 


98.0 


34 


[REDACTED] 


563 


95.7 
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Key 


80 . 0 - 89 . 9 % 


♦ 


70 . 0 - 79 . 9 % 


• 


60 . 0 - 69 . 9 % 


X 


50 . 0 - 59 . 9 % 


o 


40 . 0 - 49 . 9 % 


□ 


< 40 . 0 % 


▲ 
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Table 4.19: Summary Statistics for New Reading Test Items in Combined Sample 
of Current and Prospective Applicants 



Reading items 

Percent correct 


N 


90 . 0 - 100 . 0 % 


33 


80 . 0 - 89 . 9 % 


1 


70 . 0 - 79 . 9 % 


0 


60 . 0 - 69 . 9 % 


~o1 


50 . 0 - 59 . 9 % 


0 


40 . 0 - 49 . 9 % 


0 


< 40 . 0 % 


0 


Total 


34 
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Table 4.20: Item Properties of New Writing Tests Items in Combined Sample of 
Current and Prospective Applicants 



Item 

number 


Item text 


N 

combined 


Avg. 

percent 

correct 


Flag 


1 


[REDACTED] 


876 


75.4 


• 


2 


[REDACTED] 


712 


80.8 


♦ 


3 


[REDACTED] 


874 


89.1 


♦ 


4 


[REDACTED] 


874 


85.8 


♦ 


5 


[REDACTED] 


830 


89.4 


♦ 


6 


[REDACTED] 


700 


83.9 


♦ 


7 


[REDACTED] 


898 


85.1 


♦ 


8 


[REDACTED] 


1407 


81.5 


♦ 


9 


[REDACTED] 


1171 


81.2 


♦ 


10 


[REDACTED] 


780 


82.4 


♦ 


11 


[REDACTED] 


739 


82.9 


♦ 


12 


[REDACTED] 


644 


76.3 


• 


13 


[REDACTED] 


688 


74.5 


• 


14 


[REDACTED] 


814 


85.1 


♦ 


15 


[REDACTED] 


773 


81.2 


♦ 


16 


[REDACTED] 


784 


78.7 


• 


17 


[REDACTED] 


814 


83.0 


♦ 


18 


[REDACTED] 


790 


82.6 


♦ 


19 


[REDACTED] 


682 


71.3 


• 


20 


[REDACTED] 


800 


89.6 


♦ 


21 


[REDACTED] 


663 


79.2 


• 


22 


[REDACTED] 


840 


88.1 


♦ 


23 


[REDACTED] 


666 


76.3 


• 


24 


[REDACTED] 


797 


86.9 


♦ 


25 


[REDACTED] 


788 


84.1 


♦ 


26 


[REDACTED] 


789 


86.9 


♦ 


27 


[REDACTED] 


719 


65.5 




28 


[REDACTED] 


684 


71.0 


• 


29 


[REDACTED] 


765 


75.6 


• 


30 


[REDACTED] 


700 


77.8 


• 


31 


[REDACTED] 


759 


84.1 


♦ 


32 


[REDACTED] 


757 


81.9 


♦ 


33 


[REDACTED] 


690 


78.0 


• 


34 


[REDACTED] 


630 


69.8 


& 
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80 . 0 - 89 . 9 % 


♦ 


70 . 0 - 79 . 9 % 


• 


60 . 0 - 69 . 9 % 


X 


50 . 0 - 59 . 9 % 


o 


40 . 0 - 49 . 9 % 


□ 


< 40 . 0 % 


▲ 
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Table 4.21 : Summary Statistics for New Writing Test Items in Combined Sample 
of Current and Prospective Applicants 



Writing items 




Percent correct 


N 


90 . 0 - 100 . 0 % 


0 


80 . 0 - 89 . 9 % 


21 


70 . 0 - 79 . 9 % 


11 


60 . 0 - 69 . 9 % 


2 


50 . 0 - 59 . 9 % 


0 


40 . 0 - 49 . 9 % 


0 


< 40 . 0 % 


0 


Total 


34 
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Table 4.22: Age of Participants in Revised items Study 



Age 


N 


Percentage * 


1 8-24 years 


11 


5.1 


25-39 years 


55 


25.5 


40-49 years 


48 


22.2 


50-64 years 


80 


37.0 


65+ years 


22 


10.2 


Total 


216 


100 


* excludes 36 cases for which age information was not provided 



218 





Naturalization Test Redesign Project 
Civics Item Selection Analysis 



List of Supporting Tables 



Table 4.23: Geographic Region of Birth of Participants in Revised Items Study 



Region 


N 


Percentage * 


North America 


1 


.44 


Central America 


106 


46.3 


South America 


40 


17.5 


Caribbean 


24 


10.5 


Europe 


21 


9.2 


Asia 


36 


15.7 


Africa 


1 


.44 


Oceania 


0 


.00 


Total 


229 


100 


* excludes 23 cases for which region of birth information was not 
provided 
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Gender 


N 


Percentage * 


Female 


138 


54.8 


Male 


84 


33.3 


Total 


222 


88 



* excludes 30 cases for which gender information was not provided 



List of Supporting Tables 
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the U.S. 



Years living in the U.S. 


N 


Percentage * 


Fewer than 5 years 


68 


31.1 


5-1 0 years 


48 


21.9 


1 1-20 years 


71 


32.4 


More than 20 years 


32 


14.6 


Total 


219 


100 


* excludes 33 cases for which years living in the U.S. 


was not provided 
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Table 4.26: NRS Level of Participants in Revised Items Study 



NRS level 


N 


Percentage * 


Beginning 


9 


4.1 


Low Beginning 


47 


21.6 


High Beginning 


73 


33.5 


Low Intermediate 


50 


22.9 


High Intermediate 


26 


11.9 


Advanced 


13 


6.0 


Total 


218 


100 


* excludes 34 cases for which NRS level information was not provided 
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Table 4.27: Item Properties of Revised Civics Items 



Item 

number 


Item text 


N 


Percent 

correct 


Flag 


1 


The idea of self-government is in the first three words of the 
Constitution. What are these words? 


193 


52.8 


X 


2 


What is the economic system in the U.S.? 


194 


65.5 


• 


3 


What system of government stops one branch of government from 
becoming too powerful? 


192 


38.0 


□ 


4 


Inalienable rights are natural rights that people are born with. What 
are two of these rights in the Declaration of Independence? 


166 


31.3 


□ 


5 


Who is one of your state’s two United States Senators? 


196 


73.5 


♦ 


6 


What do you call the person who represents your district in 
Congress? 


196 


39.3 


□ 


7 


Why do some states have more Representatives than other states? 


194 


54.6 


X 


8 


Our Constitution limits powers to the Federal government. What is 
one power of the Federal government? 


193 


28.0 


□ 


9 


Under our Constitution, powers belong to the states. What is one 
power of the states? 


192 


37.0 


□ 


10 


When do the people vote for President? 


194 


85.6 




11 


Why do we say the Pledge of Allegiance? 


189 


41.3 


O 


12 


When you become a citizen, what is one promise you make? 


188 


39.9 


□ 


13 


How old do citizens have to be to vote for President? 


190 


78.9 


♦ 


14 


Who was the first President and the “Father of Our Country”? 


192 


91.1 




15 


The Federalist papers were a group of essays supporting the 
passage of the U.S. Constitution. Name one of the writers. 


185 


56.8 


X 


16 


What is one important thing that Abraham Lincoln did? 


190 


54.2 


X 


17 


What countries did the United States fight in World War II? 


190 


71.1 


♦ 


18 


Why was the United Nations created? 


188 


43.6 


O 


19 


Before becoming President, Eisenhower was a General. What war 
was he in? 


189 


49.2 


o 



Key 


70.0-79.9% 


♦ 


60.0-69.9% 


• 


50.0-59.9% 


X 


40.0-49.9% 


o 


< 39.9% 


□ 
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Table 4.28: Summary Statistics for Revised Civics Items 



Civics items 




Percent correct 


N 


80 . 0 - 100 % 


2 


70 . 0 - 79 . 9 % 


3 


60 . 0 - 69 . 9 % 


1 


50 . 0 - 59 . 9 % 


4 


40 . 0 - 49 . 9 % 


3 


< 39 . 9 % 


6 


Total 


19 
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Table 4.29: Item Properties of Revised Reading Items 



Item 

number 


Item text 


N 


Percent 

correct 


Flag 


1 


[REDACTED] 


201 


97.0 




2 


[REDACTED] 


200 


92.5 




3 


[REDACTED] 


200 


99.0 




4 


[REDACTED] 


200 


97.0 




5 


[REDACTED] 


200 


98.5 




6 


[REDACTED] 


200 


98.5 




7 


[REDACTED] 


200 


93.5 




8 


[REDACTED] 


200 


96.5 




9 


[REDACTED] 


200 


99.0 




10 


[REDACTED] 


200 


99.0 




11 


[REDACTED] 


200 


97.5 




12 


[REDACTED] 


200 


92.5 




13 


[REDACTED] 


1 51 n 


97.4 




14 


[REDACTED] 


151 


98.7 




15 


[REDACTED] 


152 


96.1 




16 


[REDACTED] 


50 


100 





Key 


70.0-79.9% 


♦ 


60.0-69.9% 


• 


50.0-59.9% 


X 


40.0-49.9% 


o 


< 39.9% 


□ 
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Table 4.30: Summary Statistics for Revised Reading items 



Reading items 

Percent correct 


N 


80 . 0 - 100 % 


16 


70 . 0 - 79 . 9 % 


0 


60 . 0 - 69 . 9 % 


0 


50 . 0 - 59 . 9 % 


0 


40 . 0 - 49 . 9 % 


0 


< 39 . 9 % 


0 


Total 


16 
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Table 4.31 : Item Properties of Revised Writing Items 



Item 

number 


Item text 


N 


Percent 

correct 


Flag 


1 


[REDACTED] 


218 


70.2 


♦ 


2 


[REDACTED] 


218 


80.3 




3 


[REDACTED] 


220 


78.2 


♦ 


4 


[REDACTED] 


220 


75.0 


♦ 


5 


[REDACTED] 


220 


70.0 


♦ 


6 


[REDACTED] 


205 


51.7 


X 


7 


[REDACTED] 


197 


76.6 


♦ 


8 


[REDACTED] 


198 


69.2 


• 


9 


[REDACTED] 


196 


75.0 


♦ 


10 


[REDACTED] 


197 


74.6 


♦ 


11 


[REDACTED] 


195 


77.4 


♦ 


12 


[REDACTED] 


196 


68.4 


• 


13 


[REDACTED] 


178 


65.2 


• 


14 


[REDACTED] 


112 


74.1 


♦ 


15 


[REDACTED] 


112 


56.3 


X 


16 


[REDACTED] 


53 


58.5 


X 



Key 


70.0-79.9% 


♦ 


60.0-69.9% 


• 


50.0-59.9% 


X 


40.0-49.9% 


o 


< 39.9% 


□ 
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Table 4.32: Summary Statistics for Revised Writing Items 



Writing items 

Percent correct 


N 


80 . 0 - 100 % 


1 


70 . 0 - 79 . 9 % 


9 


60 . 0 - 69 . 9 % 


3 


50 . 0 - 59 . 9 % 


3 


40 . 0 - 49 . 9 % 


0 


< 39 . 9 % 


0 


Total 


16 
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Table 4.33: Difference in Percent Correct Between Civics Items From 
Prospective Applicants and Revised Civics Items 



Prospective applicants 

Item 


Percent 

correct 


Revised items study 

Item 


Percent 

correct 


Difference 


131. What does "We the People" 
mean in the Constitution? 


13.9 


1 . The idea of self-government is 
in the first 3 three words of the 
constitution. What are these 
words? 


52.8 


+38.9 


1 41 . What type of economic system 
does the U.S. have? 


23.9 


2. What is the economic system 
in the U.S.? 


65.5 


+41.6 


93. Name your state's two U.S. 
Senators. 


42.9 


5. Who is one of your state's two 
U.S. Senators? 


73.5 


+30.6 


84. Who does a U.S. Representative 
represent? 


22.5 


6. What do you call the person 
who represents your district in 
Congress? 


39.3 


+16.8 


85. What decides each state's 

number of U.S. Representatives? 


24.8 


7. Why do some states have 
more Representatives than 
other states? 


54.6 


+29.8 


74. Name one example of checks 
and balances. 


16.0 


3. What system of government 
stops one branch of 
government from becoming too 
powerful? 


38.0 


+22.0 


64. Name one thing only the federal 
government can do. 


19.4 


8. Our Constitution limits powers 
to the Federal government. 
What is one power of the 
Federal government? 


28.0 


+8.6 


1 1 6. What is one thing a state 
government can do? 


21.2 


9. Under our Constitution, powers 
belong to the states. What is 
one power of the states? 


37.0 


+15.8 


55. In what month are general 

presidential elections held in the 
United States? 


73.0 


1 0. When do the people vote for 
President? 


85.6 


+12.6 


22. What are "inalienable rights"? 
AND 


5.9 


4. Inalienable rights are natural 
rights that people are born 
with. What are two of these 
rights in the Declaration of 
Independence? 


31.3 


+25.4 


12. Name one important idea found 
in the Declaration of 
Independence. 


21.6 


+9.7 


42. What is the Pledge of 
Allegiance? 


10.8 


1 1 . Why do we say the Pledge of 
Allegiance? 


41.3 


+30.5 


127. Name one promise you make 
when you say the Oath of 
Allegiance. 


26.3 


12. When you become a citizen, 
what is one promise you 
make? 


39.9 


+13.6 


75. Who can vote in the U.S. 
presidential elections? 


21.0 


1 3. How old do citizens have to be 
to vote for President? 


78.9 


+57.9 


1 23. Who is called the "Father of Our 
Country"? AND 


74.4 


14. Who was the first President 




+16.7 


47. Who was the first President? 


89.1 


and the "Father of Our 
Country"? 


91.1 


+2.0 
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Item 


Percent 

correct 


Revised items study 

Item 


Percent 

correct 


Difference 


2. Name one of the writers of the 
Federalist Papers? AND 


8.5 


15. The Federalist papers were a 
group of essays supporting the 




+48.3 


86. What group of essays supported 
passage of the U.S. 
Constitution? 


9.6 


passage of the U.S. 
Constitution. Name one of the 
writers. 


56.8 


+47.2 


111. Name one of the things that 
Abraham Lincoln did. 


57.1 


16. What is one important thing 
that Abraham Lincoln did? 


54.2 


-2.9 


94. The United States fought Japan, 
Germany, and Italy during which 
war? 


62.8 


17. What countries did the United 
States fight in World War II? 


71.1 


+8.3 


126. What international organization 
was established after World War 
II (WWII) to keep the world at 
peace? 


31.9 


18. Why was the United Nations 
created? 


43.6 


+11.7 


46. Which U.S. World War II general 
later became President? 


15.3 


19. Before becoming President, 
Eisenhower was a General. 
What war was he in? 


49.2 


+33.9 


Average percent correct 


31.5 


Average percent correct 


54.3 


+22.8 



230 





Naturalization Test Redesign Project 
Civics Item Selection Analysis 



List of Supporting Tables 



Table 4.34: Difference in Percent Correct Between Reading Items From 
Prospective Applicants and Revised Reading Items 



Prospective applicants 




Revised items study 






Percent 




Percent 


Item 


correct 


Item 


correct Difference 


6. [REDACTED] 


94.3 


1 . [REDACTED] 


97.0 +2.7 
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Table 4.35: Difference in Percent Correct Between Writing Items From 
Prospective Applicants and Revised Writing Items 



Prospective applicants 


Revised items study 






Percent 


Percent 


Item 


correct Item 


correct Difference 


6. [REDACTED] 


71.9 14. [REDACTED] 


74.1 +2.2 
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Table 5.1 : Items From Sample of Current Applicants With Low Item 

Discrimination 



Item text 


Item discrimination 


Civics items 


Who lived in America before the Europeans arrived? 


0.24 


Who is the President now? 


0.09 


Who makes federal laws? 


0.24 


What is the capital (or capital city) of your state? 


0.23 


We elect a President for how many years? 


0.16 


Who was the first President? 


0.15 


When do we celebrate Independence Day? 


0.14 


Why does the flag have 13 stripes? 


0.16 


There are 13 original states. Name three. 


0.22 


Who is called the “Father of Our Country”? 


0.16 


What is the highest court in the U.S.? 


0.11 


On the Fourth of July we celebrate independence from what country? 


0.23 


Reading items 


[REDACTED] 


-0.02 


[REDACTED] 


0.23 


[REDACTED] 


-0.01 


[REDACTED] 


0.00 


[REDACTED] 


0.00 


[REDACTED] 


-0.01 


[REDACTED] 


-0.01 


[REDACTED] 


-0.01 


[REDACTED] 


0.23 


[REDACTED] 


0.24 


[REDACTED] 


0.22 


Writing items 


[REDACTED] 


0.13 


[REDACTED] 


0.24 


[REDACTED] 


0.22 


[REDACTED] 


0.19 


[REDACTED] 


0.18 


[REDACTED] 


0.13 
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